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ABSTRACT 



Data from the California Development Department's 
Unemployment Insurance System were used to track the earnings and employment 
of more than 57,000 California Employment Training Panel (ETP) trainees 
served by ETP projects that ended in 1994-95 and 1995-96. ETP trainees' 
earnings gains averaged 15%, versus 11.8% in the case of similar workers who 
were not trained in an ETP program. Companies that participated in ETP grew 
faster than nonparticipating companies did (a growth rate of 14% versus 
-.8%) . The $73 million spent by the ETP directly on training contracts was 
determined to have had an economic impact on California's economy estimated 
at more than $400 million. It was recommended that ETP take the following 
four actions: (1) continue to target its resources toward basic industries 
and companies that are below industry best practices; (2) encourage 
management reinforcement of training; (3) increase the amount of customized 
employer -provided training and reduce the amount of generic, training agency 
training; and (4) continue to attempt to measure the impact of ETP training 
on individual companies. (Contains 17 references and 42 tables/figures. A 
comparison of trainees before 1994, in 1994, and after 1994 and regression 
results are appended.) (MN) 
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Training That Makes A Difference: 

ETP’s Impact on Trainees, Companies and California’s Economy 



Executive Summary 



Overview 

The California Employment Training Panel (ETP) was established in 1982 to ensure that 
employers had the trained workers they needed to compete in a global economy. The Panel 
contracts with individual employers, groups of employers, and training agencies to provide 
the training it funds. 1 

This is the second report in a comprehensive evaluation of ETP’s effectiveness. The first 
report of the study, ETP AT Work, summarized the results of 23 case studies, which 
examined the dynamics of ETP in the field and identified factors that affect ETP’s impact on 
companies served. This second report presents the results of a large-scale statistical study of 
ETP’s impact from three perspectives: 

1. ETP’s impact on individual trainee’s earning and employment patterns, 

2. ETP’s impact on company growth, 

3. ETP’s overall impact on the State’s economy. 

Methods 

This study used data from California Employment Development Department’s 
Unemployment Insurance System to track the earning and employment of over 57,000 ETP 
trainees served by projects that ended in 1994-95 and 1995-96. The analysis used data from 
the same system to track employment and earnings growth within companies served by ETP 
The experience of both individuals and companies served by ETP were compared with 
control groups of similar individuals and companies not served by ETP to estimate the 
impact of ETP. ETP’s impact on the state's economy was estimated based on its impact on 
individuals and companies. 

Results 

Overall ETP trainees had greater earnings gains and more stable employment than similar 
workers who were not trained. Companies who participated in ETP grew faster than 
companies that did not participate. Finally, ETP was found to have a positive impact on the 
state’s economy, which far exceeded its costs. 

The experience New Hire trainees (trainees who were unemployed when they entered 
training) and Retrainees (employed workers who were trained by ETP) were examined 



1 Employment Training Panel Strategic Plan 1998-2001. 
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separately, and the experiences of trainees who dropped out and completed were compared as 
well. All trainee groups were measured against control groups of similar workers. 



Both New Hires and Retrainees were more likely to remain in the California labor force after 
training than similar workers in the same industries. This study, unlike previous studies, 
found that dropouts and completers persisted in the labor market at the same rate, probably 
due to the current boom economy. Both New Hire and Retrainee completers had roughly 
equal earnings gains of about $3,600 or 15% over two years, compared to about $3,000 and 
an 1 1.8% increase for similar workers who were not trained by ETP. Again, dropouts did 
nearly as well as completers due to the prosperous economy. 

Both New Hires and Retrainees experienced a sharp decline in the weeks they were 
unemployed after training. Overall we found ETP trainees had much more stable 
employment after training than similar workers. This meant they were unemployed less and 
were less likely to change industries. In this case completers differed from dropouts in that 
they had substantially greater increases in employment stability. 

Companies who participated in ETP training grew faster than similar companies who did not 
participate. Employment in companies served by ETP grew over 14% from the year before 
to the year after training, compared to a small decline of -.8% for similar companies. Total 
payroll in ETP served companies grew by almost 26% compared to about 10% for similar 
companies. How fast companies grew was related to the type of training contract. 

Companies who contracted directly with ETP grew fastest, followed by companies that were 
served through consortia; companies served by training agencies grew slowest. The size of 
the training intervention seemed to make a difference as well; companies that trained over 
10% of their workers grew substantially faster than companies who trained less than 10% of 
their workers. 



The analysis estimated that the $73 million ETP spent directly on training contracts ending in 
1994-95 and 1995-96 had an over $400 million impact on the State’s economy in the year 
after training. The outcomes of ETP training had an impact on California’s economy three 
ways: by increasing trainees’ employment stability, increasing trainees’ productivity, and 
saving California Jobs. Our estimates of each effect are: 



Training Outcomes 

1 . Increased employment stability for trainees 
S Unemployment Insurance fund savings 

2. Increased productivity of trainees 
S Increased trainee earnings 

S Increased sales by other California businesses 

3. Saving California jobs 

S Unemployment Insurance fund savings 
S Prevented temporary earnings losses by trainees 
S Prevented losses by other California businesses 



Estimated Benefits 



$2 million 

$33 million 
$17 million 

$61 million 
$167 million 
$134 million 



TOTAL 



$414million 
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Recommendations 



• Increasing ETP’s Impact on California’s Economy 

1. ETP must strive to continually improve the quality of training offered through its 
contracts. 

Effective training is the foundation for ETP’s impact on the California economy. The 
effectiveness of the training determines whether or not trainee productivity increases. It is 
through increased productivity that ETP enhances California companies’ competitiveness, 
saves California jobs, raises workers’ earnings, and increases their employment stability. 

2. ETP must continue to target its resources on basic industries. 

Increasing the productivity and competitiveness of California’s basic industries leverages 
ETP’s impact on the California economy. When these businesses grow they generate more 
business for other California companies and more jobs for California workers. Serving these 
basic industries should remain a priority. 

3. ETP should increase the proportion of its funds targeted on retrainees threatened with 
displacement 

Saving California jobs produced ETP’s largest monetary impact on the California economy; 
therefore it should be ETP’s top priority. Had the jobs that were threatened by out-of-state 
competition been temporarily lost for just the length of the average unemployment episode, 
California would have lost over $362 million in additional unemployment claims, decreased 
business and earnings, and negative indirect effects on California suppliers. ETP should 
increase the proportion of its funding targeted on potentially displaced workers. 

• Increasing ETP’s Impact on Companies 

4. Target ETP training on companies that are below industry best practices. 

Companies whose operations were far below industry standards had the greatest potential 
gains from ETP training. Similarly, workers in these low performing companies are most at 
risk for losing their jobs, and saving those jobs is the most efficient way for ETP to have a 
positive economic impact on the state. Thus targeting ETP funds on companies whose 
practices are substantially below their industries’ best practices will increase ETP’s impact. 

5. ETP should encourage management reinforcement of training. 

Even poorly planned, low-quality training can have a positive impact on companies if 
management provides strong reinforcement of the skills and techniques learned. Conversely, 
well-planned, high-quality training can fail to yield positive results if there was not 
systematic management reinforcement. ETP should use its contracting and monitoring 
processes to promote management reinforcement of training. 
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6. Increase the amount of customized employer-provided training and reduce the amount 
of generic, training-agency training. 

This study reveals that the most effective training model is the employer contract where a 
company provides customized training to its own workers. Our analysis shows that in recent 
years, ETP funding has shifted away from the customized, employer-provided training to the 
generic skills training agency contracts. For example, training agency contracts served 20% 
of all trainees in 1994-95. In 1995-96 they served 30% of all trainees, a growth rate of 50%. 
We found the generic skill training which employees do on their own time to be the least 
effective. If ETP wishes to increase the impact of ETP training on companies we 
recommend this trend be reversed. Thus we make the following specific recommendations: 

6a. Do not fund training where employers are not paying a significant share of 

the costs. 

6b. Phase out generic skills training over two years and increase the focus on 
industry specific skills training. 

7. ETP should not require that a minimum or maximum proportion of a company’s 
workers be trained 

While we found a limited positive association between the percent of workers trained and the 
impact on companies, the increase in impact was not directly proportional to the increase in 
employees trained. It does not appear that there is a minimum proportion of workers that 
must be trained for ETP to have a positive impact, nor does there appear to be a point at 
which the impact of ETP training dramatically diminishes as the proportion trained rises. 
Hence we do not recommend setting a minimum or maximum proportion of workers to be 
trained. This is probably a decision best left to employers. 

8. ETP should continue to attempt to measure the impact of ETP training on individual 
companies. 

Despite a widespread interest in the impact of training on company performance there are 
few published studies. This study showed that it is possible to obtain valid and reliable data 
on company growth, which can be accurately benchmarked against similar companies. We 
recommend that ETP explore ways to collect the company growth data on a regular basis for 
ETP companies and a comparison group to provide a regular method of monitoring ETP’s 
impact on the companies served. 
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Introduction 



The California Employment Training Panel (ETP) was established in 1982. It is supported by a 
special employment training tax, which is levied on the California employers who participate in 
the Unemployment Insurance system, at a maximum of $7 per employee annually. ETP expects 
revenues of $102.1 million in 1998-99, of which $90.7 million will be available for training. The 
Panel contracts with individual employers, groups of employers, and training agencies to provide 
the training it funds. 1 

Two recent nation-wide studies of state-run incumbent worker training programs, one from the 
National Governor’s Association and one from the U.S. Department of Labor, recognize that 
ETP is the largest and most sophisticated program of its type in the nation. Both reports also 
note that serious evaluation studies of incumbent programs are sorely lacking. ETP is the 
exception. It has subjected itself to periodic independent evaluation of its impact on both 
individual trainees and the state’s economy repeatedly over its history (Duscha and Graves, 

1999; Regional Technology Strategies, 1999). This report presents the most recent 
comprehensive evaluation of ETP ’s impact on the workers trained, the companies that employ 
them, and the state's economy as a whole. 

This study is designed to evaluate the performance of the California Employment Training Panel 
in relationship to its mission, which is: 

ETP Mission Statement 

The Employment Training Panel is a significant economic development tool for business 
attraction and business retention. ETP will work in partnership with business, labor, and 
government to provide funds for training California ’s workforce in the skills necessary 
for businesses to remain viable and compete in the global economy, while providing 
workers with reasonable wages and secure employment . 2 

The mission shows that the ETP program is designed to have impacts at three different levels. 
First the program aims to increase the employment stability and incomes of individual workers 
trained under the program. Next, the program aims to increase the competitiveness of California 
companies in basic industries. 3 Finally, the program is intended to have a positive impact on 
overall California economy. In this component of the evaluation we attempt to measure the 
success of ETP in program years 1994-95 and 1995-96 in achieving these goals. 



| Employment Training Panel Strategic Plan 1998-2001. 

' Employment Training Panel Strategic Plan 1998-2001. 

3 A basic industry is one that exports most of its products to customers outside the region in question. The essential 
notion of a basic industry is that it is not dependent on local demand for its business and that additional activity in 
basic industries will spawn even more local activity since the basic industry buys many of its inputs locally. 
Consequently, growth in basic industries will spur even more growth in the local economy. 
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Is there Underinvestment in Training? 

ETP’s very existence assumes that private employers are underinvesting in training their 
workers, or else there would be no need for ETP intervention. In fact, there is mounting 
evidence that American employers do underinvest in worker training and further evidence that 
California employers often invest even less than the national average. A number of researchers 
have compared U S. employers’ patterns of investment in training with similar companies in 
Europe and Japan and concluded that the U.S. companies are underinvesting (see for example 
Lynch, 1993 and Bishop, 1995). Underinvestment seems to be particularly acute when it comes 
to the non-college educated workers, who make up the large majority of the workforce. Lynch 
(1993, p.2) sums up the situation this way: 

Underinvestment in training in the U.S. appears to be of two forms. First, in 
certain sectors, U.S. firms may be spending less and providing more limited 
training to their non-technical or non-managerial employees than their 
competitors in other countries. Second, in other sectors the level of expenditures 
or hours of training may be the same, but due to lower initial skill levels, this level 
of investment is not sufficient to achieve the same degree of skill proficiencies 
found in countries such as Japan or Germany. 

Bishop (1995) examines in detail a number of specific barriers to employer investment in 
training and concludes that high turnover rates in most U.S. companies yields lower 
returns on training investment, decreasing employers’ investment below what would be 
optimal from a public point of view. 

Lester Thurow, economist and Dean of MIT’s Sloan School of Management, reviewed the 
current state of the American economy and notes the underinvestment in training by both 
companies and individual workers. He explained the dynamic that leads to underinvestment this 
way. 

The basic problem in the United States is that every employer wants a free ride in 
the training system. “You train, I’ll hire” is the American way. Whenever 
unemployment is low, employers who themselves do no training, bitterly 
complain about the shortage of trained workers. They see nothing strange about 
their complaints. As for employees, without career ladders they can not 
intelligently acquire the right skills on their own. Since they will be switching 
employers frequently, they do not know which skills they will need or how long 
those skills will be relevant to their earning opportunities. As a result — 
rationally — they don’t invest in skills (Thurow, 1999, p.69). 

Not only do employers tend to underinvest in training in general, but they particularly 
underinvest in training workers who are not in professional or technical jobs. Generally, 
employers invest the large bulk of their training dollars in college educated workers. For 
example, a recent comprehensive survey of employer provided training by the Bureau of Labor 
Statistics found that only 60% of workers with a high school education or less received formal 
training from their employer in the last year, compared to 90% of employees with a bachelors 
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degree or higher levels of education. The study also found the amount of training varied 
substantially. On average those with a high school degree or less received 10.9 hours of formal 
training compared to 16. 1 hours for those with a bachelor degree or higher (Frazis, et. al., 1998). 

California businesses tend to invest even less than the national average. In a statewide survey of 
a representative sample of California employers with over 20 employees, researchers at UCLA 
found that California employers were even less likely than employers nationally to invest in 
training. In all six categories of training studied California lagged behind the nation for 
companies with over 250 employees. For example, nationally 58% of companies with over 250 
employees said they provided training in “production skills,” but only 32% of similar California 
employers said they did the same. The results for companies with between 50 and 249 
employees were more mixed, but only in the area of computer skill training was California 
consistently ahead of the nation (Erickson et. al., 1998). 

In short, available research indicates that the need for ETP to serve as an incentive for employer 
investment in training frontline workers remains strong. 



Benchmarking the Impact of ETP Training 

In evaluating the impact of ETP training the question always arises: How much of an impact is 
enough? We consulted available research on the impact of employer-provided training against 
which to benchmark the performance of ETP Retraining programs, and consulted the research on 
displaced worker retraining to benchmark ETP’s New Hire training. 

• Retraining Benchmarks 

Human capital theory suggests that training workers will make them more productive and 
increased productivity will lead to increased earnings (Becker, 1975). Research studies support 
this view and have found a positive impact on workers’ earnings when they receive employer- 
provided training (see for example Lillard and Tan, 1985; Mincer, 1989; Hollenbeck and Wilkie, 
1985; Lengermann, 1996). These impacts range from about a 5% to 12% increase in earnings, 
and the results vary with the population studied and how earnings were measured. Previous 
research on ETP found that the impact of ETP training was similar to that of private employer- 
provided training and much greater than other public training programs such as JTPA (Moore, 
Blake and Phillips, 1995; Moore, Blake and Phillips, 1994). The question for this study is: Has 
the impact of ETP training remained at the level of private employer-provided training during the 
current period of economic growth? 

• New Hire Benchmarks 

Two sophisticated studies of displaced worker retraining programs provide valuable benchmarks 
for evaluating ETP New Hire training, one done in Texas (Bloom, 1990) and one in New Jersey 
(Anderson, et. al., 1990; Corson, et. al., 1989). These studies used an experimental design and 
found impacts of only several hundred dollars in the first year after receiving services. The New 
Jersey study (Corson, et. al., 1989) examined three different types of services: job search, job 
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search and a cash bonus for finding a job, and job search with training. Surprisingly, the 
treatment that involved training had the smallest impact on earnings, about $200, of all the 
services tested. In a longer-term follow-up of these workers (Anderson, et. al., 1990) researchers 
found none of the three groups had earnings gains that exceeded inflation. The Texas study 
(Bloom, 1990) found substantially larger impacts, ranging from $673 for men to $1,148 for 
women, who received a mixture of job search and short-term skill training services. 

Recent data on changes in earnings from the year before to the year after training for participants 
in California’s JTPA title III program provide another benchmark. According to this analysis, 
workers who completed training had median earnings gains of $5,161 and those who left without 
completing experienced median gains of $1,606. Suggesting that the value of completing 
training is $3,555 (State Job Training Coordinating Committee, 1998). 



Overview of this Study 

We begin by examining the impact of ETP training on the individual workers served. We do this 
by examining the trainees’ attachment to the labor force, their individual earnings before and 
after training, and patterns of employment and unemployment before and after training. ETP 
trainees are compared with a random sample of California workers (the Control Group) who 
have been matched to ETP trainees based on their industry of employment. Also, ETP trainees 
who completed training (called completers) are compared with trainees who did not complete 
training (called dropouts). Finally, ETP trains two types of trainees: Retrainees, who are 
currently employed workers, and New Hires, workers who are unemployed and who are 
collecting Unemployment Insurance or who have exhausted their benefits. The experiences of 
these two groups will be examined separately. That is, the experience of Retrainee completers 
and dropouts will be compared to that of the Control Group and with one-another. The same 
comparisons will be made for the New Hire completers and dropouts. 

Next we turn our attention to the impact of ETP on the companies which employed the trainees, 
by benchmarking the growth of these companies against that of other similar companies not 
served by ETP. 

Finally we estimate the impact of the entire ETP program for the years studied on the overall 
California economy. 
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Methods 



Trainee Analysis 

• Objectives 

The trainee analysis had three objectives: 

1 Measure the labor force attachment of trainees in the year before training and the two 
years after training in comparison with the Controls. 

2. Measure the change in earnings and unemployment for those trainees who remained in 
the labor force through out the two-year follow-up period, again in comparison with the 
Controls. 

3. Measure the employment stability of trainees before and after training, in terms of their 
level on unemployment and stability within a particular industry, in comparison with the 
Controls. 



• Trainee Population 

We begin the analysis with a discussion of the characteristics of ETP’s trainee population. 

The population we studied includes all trainees who were served by ETP projects which ended in 
fiscal years 1994-95 and 1995-96, and it included both those who completed training and those 
who dropped out. Because of the length of time ETP projects take to complete, this study 
includes projects, which began as far back as 1992-93. In total we included 323 projects and 
57,473 participants in the analysis. 



- studv Pcipulation *1 


BlllliO! 


Trainees 


Projects Closed 1 994-95 
(No. of Projects =214) 


Projects Closed 1995-96 
(No. of Projects =109) 


Total 

Number 


Total 

Percentage 


Retrainees Total 


34,780 


18,350 


53,130 


92.4% 


Completers 


27,441 


15,132 


42,573 




Dropouts 


7,339 


3,218 


10,557 




New Hire Total 


2,347 


1,996 


4,343 


7.6% 


Completers 


1,031 


1,097 


2,128 




Dropouts 


1,316 


899 


2,215 




Total Trainees 


37,127 


20,346 


57,473 


100.0% 
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The period studied was one of rapid economic change. The California economy was emerging, 
late, from the national recession of the late 1980’s and early 1990’s. This was a period of rapid 
economic growth in which job creation was slow at first but then accelerated. We were 
concerned that differences in the changes in earnings and employment might be attributable to 
the period in which observations were made rather than the impact of training. For example, a 
trainee exiting a program in 1993 when employment was growing slowly might not have the 
same opportunities or wage increases as a trainee who exited in 1995 when the employment was 
growing rapidly. To control for the shifting economic conditions that occurred during the time 
period we studied, we divided the trainees into three groups based on whether they exited 
training before 1994, during 1994, or after 1994. We compared the experiences of the trainee 
groups to those of three corresponding Control Groups, which were random samples of 1% of all 
California workers drawn from the second quarters of 1993, 1994, and 1995. We then followed 
all trainees for two full years after exiting the program. The results of this analysis can be found 
in Appendix A. Our analysis showed that the experience among the three groups of trainees 
were not dramatically different so they were merged back together for the analysis presented 
here. 

Figure 1 shows the distribution of all ETP trainees in the study across industry groups in 
comparison with a random sample of one percent of all California workers. ETP trainees were 
assigned to an industry group based on the SIC code of their employer during training. The 
Control Group members were assigned an industry groups based on the SIC code of their 
employer in the quarter in which they were sampled. This analysis provides a quick profile of 
the industries served by ETP and their relationship to the larger California economy (represented 
by the Control Group) The graph shows that the three industries with the largest number of ETP 
traineees were: electronic and electrical equipment and components manufacturing; measuring 
and related equipment and miscellaneous manufacturing, and retail trade. By and large this 
distribution reflects ETP’s focus on basic industries, particularly high technology manufacturing. 
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SIC Groups 



Figure 1 

SIC Group Distribution of the Trainees and Controls 
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• Measurements: The Data 



The data on the number and characteristics of the trainees were taken from ETP trainee and 
contract files. Their earnings and employment experiences were extracted from the Base Wage 
File and Single Client Database which are maintained by the California Employment 
Development Department. These earnings and unemployment data pertain to Unemployment 
Insurance (UI) covered jobs in California. There are a number of jobs in California that are not 
included in the Base Wage File. Federal employees, military personnel, and the self-employed 
are the most prominent of these, but there is a miscellaneous group of other California jobs not 
covered by the UI Base Wage File. People employed in these jobs will not show up in the 
databases we used to track Trainees and Controls. The reader should be cautioned that all of our 
results concerning labor force participation, earnings, and unemployment claims are conditioned 
on being in Ul-covered employment in California. Obviously, Trainees and Control group 
members who find jobs in non-UI-covered employment will disappear from our databases. 

S Labor Force Attachment 

Trainees and Controls were classified as in the labor market if they had either $1 of earnings or a 
UI claim in any given quarter. Attachment is reported as the percent of trainees or Controls who 
met this criterion. This analysis was not restricted to trainees found in the workforce the fourth 
quarter before training, fourth quarter after training and eighth quarter after training. Essentially 
we calculated the percent of the 57,473 trainees who were in the labor market each quarter. 
Aggregate trainee earnings and unemployment claims, used in the economic impact analysis, 
were also taken from this unrestricted database. Another measure of attachment is the 
percentage of trainees found in the 4 th quarter before training, and the 4 th and 8 th quarter after 
training. This measure is a byproduct of the earnings comparisons and is discussed below. 

S Earnings 

Individual earnings were based on earnings reported quarterly in the Base Wage File by 
employers to the California Employment Development Department as part of the Unemployment 
Insurance Tax collection system. To calculate an individual’s quarterly earnings, all wages and 
salaries reported by any employer for a quarter were totaled. Aggregate earnings and 
unemployment data, used in the company impact analysis, were taken from EDD’s ES 202 File. 
All earnings were adjusted for inflation to 1995 dollars. 

S Unemployment Insurance Claims 

The UI single Client Database in the California Employment Development Department provided 
the number of weeks in each quarter that an individual claimed Unemployment insurance 
compensation and the dollar amount paid each individual in Unemployment Insurance claims. 
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• Measurements: The Year Before and after Training Period 

ETP projects may begin any time and run from six months to two years. In order to define the 
year before and after training, we calculated a unique year for each trainee using the enrollment 
date and the completion or dropout date. From the enrollment date we went back in time to the 
end of the last complete quarter and then counted back four quarters to make the year before. 

For the post-training year we took the exit date, went forward to the beginning of the first 
complete quarter, then counted forward four quarters, to define the year after training and then to 
the eighth quarter after training to define the second year after training. 



• Measurements: Earnings and UI Comparisons 



To make before and after earnings and other comparisons, we defined a group of trainees who 
were attached to the California labor market for the entire period. Specifically, we selected the 
subgroup of trainees and Controls who were attached to the California labor market the fourth 
quarter before training and the fourth quarter and eighth quarter after training because they 
reported either wages or UI payments in those quarters. The figure below shows these selection 
criteria. This criteria assumes that if a trainee was in California the 4 th quarter before training, 
and was here obviously for the training, that is possible to calculate an accurate year before 
training measure. If a trainee is not found in one of the three quarters between the 4 th quarter 
before training and the training period, a “0” is assumed. Similarly, if a trainee is in the 
California labor market in the 4 quarter after training and the 8 th quarter after training, but 
missing in intervening quarters we assume a “0” value for those quarters. 



If a trainee is not present in the critical (-4, +4, +8) quarters he or she is dropped from the 
analysis because we can not confidently assume the individual was in the Ul-covered California 
labor market for the period under study. For example, a trainee who is found the 4 th quarter 
before but not in the 4 th or 8 th quarter after may have left the labor force, because she had a child 
and stopped working, went to college, joined the military, died, or moved out of state to work. 
Also, the person may have moved to a job in California that is not covered by UI and therefore 
not in the UI database. Hence, to assume 0 earnings would be false, and underestimate the real 
average earnings of the group. Conversely to include only those workers found in every quarter, 
would falsely overestimate the average earnings because no trainee would ever be unemployed. 
The method we chose allows us to assume some quarters of 0 earnings, but reasonably ensure 
that all subjects were in the California labor market for the period under study. 



Selection Criteria for Trainee Labor Market Outcomes Analysis 
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• Control Group Population and Database Adjustments 

We selected three Control Groups by taking one percent of all California workers reporting 
wages or UI claims in the 2nd quarter, 1993, 1994, and 1995. These Control Groups were then 
used to benchmark trainee changes in wages, UI payments, and UI weeks during the various 
periods of the economic recovery in California in the final analysis. 

Because we used the second quarter to draw each Control sample, the quarterly data for the 
Control Groups showed a significant seasonal pattern, which was not present in the trainee data 
The Control Group data were seasonally adjusted to purge this pattern and reveal longer-term 
trends. To do this we smoothed the quarterly data by averaging a quarter with two quarters 
before and one quarter after to generate a rolling quarterly average; this provided a more valid 
benchmark against which to judge the trainees’ experiences. 

The ETP trainees represented a different mix of industries than the Control Group as revealed in 
Figure 1. To adjust for this difference, we assigned all trainees to one of 25 industry groups. We 
also divided the Control Groups into the same 25 industry clusters. We calculated outcomes for 
all 25 industry clusters within the Control Group, and we then weighted the outcomes for each 
cluster within each of the three annual Control Groups to match the industrial mix of trainees in 
each of the three trainee cohorts. The industry-weighted outcomes from these three Control 
Groups were compared to that of the corresponding trainee groups. Since the differences 
between the trainee groups and the corresponding annual Control Groups were very similar, we 
report the differences between all trainees and “composite” Control Group made up of all three 
annual groups. To obtain the composite Control Group, we weighted the outcomes of the three 
Control Groups a second time to match the size of the corresponding trainee groups. Thus, the 
composite Control Group represents a group of workers from the same years and industries as 
the trainees. Similarly the overall Control Group is representative of the same industry mix as 
the entire group of trainees. 

Finally, in the earnings and UI comparisons we used the same criterion for the Controls that we 
applied to trainees, (-4, +4, +8) to ensure that the Controls selected were in the California labor 
market for the period under study. 

Economic Impact Analysis and Company Analysis 

Details on the methods used to evaluate ETP’s impact on the state’s economy, and ETP’s impact 
on individual companies are provided along with the results of these analyses in later sections. 
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Results 



Trainee Results 

As we noted earlier in the methods section, we first examined the trainees as three separate 
annual cohorts with three separate Control Groups. We compared the results of the three annual 
cohorts and found their experiences to be similar (complete results for each cohort are an 
appendix). For simplicity the results for all three groups of trainees are presented together as a 
single group for all earnings and employment measures. The results reported for the Control 
Group are a weighted average of the experience of the three annual Control Groups. 

• Completing Training 

Completing training is the first measure of a successful program. ETP only pays for trainees 
who complete all training and are retained on a related job for 90 days. Dropouts are any 
trainees who do not complete all training and the required 90-day retention period. It is ETP’s 
policy not to officially enroll trainees until they have completed 20% of the planned training or 
40 hours. Therefore, one can assume that all dropouts have received some of the planned 
training. Thus, dropouts do not represent a lost investment for ETP. But they do represent an 
administrative problem for ETP in that money is encumbered for candidates who enter training 
and do not complete. They also cost the ETP staff time that went into developing the project and 
monitoring it. 

Dropouts represent a real cost to employers because ETP does not reimburse any costs for the 
training of employees who do not complete training. However, the extent of the employers’ 
costs will differ by the reasons that the dropout occurred. Some dropouts are workers who were 
enrolled in training and completed a minimum amount of training because unexpected new 
orders for output required these workers return to production full time. Other dropouts may have 
actually taken part in more but not all of the training before being recalled to production by the 
employer. Obviously, some dropouts are people who quit their jobs or were laid-off during 
training or after training but before end of the 90-day certification period. All of these dropout 
cases represent some costs for the employer, the extent of which depends on how far the trainee 
got in the training process. At a minimum, the employer bears the costs of developing the 
training plan and shepherding the ETP contract through the processes for approval for trainees 
who enroll but only attend a minimal training. Generally, employers bear real costs for dropouts 
but in some cases those costs may be offset by increased profits derived from unexpected new 
orders. In those cases, even the rationale for training may disappear as a previously difficult 
competitive situation for the company melts away in the face of new orders by customers. These 
situations certainly occur more frequently in a period of economic recovery, and can be expected 
to substantially affect the labor market experience of the dropouts. 

A final dropout cost for the trainees and the employer is that dropouts may represent unrealized 
potential gains because one expects that ETP training will have a positive impact on employee 
and company productivity. Workers and employers who drop training will not make those 
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expected gains. However, dropouts and their employers may receive partial benefits from the 
training since many dropouts complete some part of the process. Also, companies that have 
initiated training contracts with ETP and then receive unexpected new orders must balance the 
expected ETP training gains against the profitability of the new orders. The unexpected new 
orders may offer profitable and productive new opportunities for the company and its workers. 
Hence, in comparing completers and dropouts one should remember that partial training can 
have a positive impact on dropouts, and that in good times dropouts may be in companies with 
new and productive opportunities. 

As the graphs below indicate, in the cohort of Retrainees studied here 80. 1% completed. This 
fits with the previous study showing that about 80.4% of Retrainees complete training (Moore, 
Blake, Phillips, 1995). However, only 49.0% of New Hires completed training, which is below 
the 56.0% completion rate reported for the 1990-91 cohort studied in the earlier ETP evaluation 
(Moore, Blake, Phillips, 1995). 



Figure 4 

Retrainee Completers vs. Dropouts 
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New Hire Completers vs Dropouts 
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• Labor Force Attachment 



Labor force attachment is calculated as the percent of trainees and Controls who were found 
either employed in a UI covered job or collecting UI benefits in any of the four quarters before 
training or eight quarters after training. As the following graphs indicate quarterly labor force 
attachment data show that the Retrainees were significantly more attached to the labor force than 
the Control Group. Immediately after training the gap was to 2%, but then begins to increase 
reaching 6% in the eighth quarter. These results indicate that over time Retrainees are more 
likely to persist in the labor force than most workers are. Interestingly, there was almost no 
difference between the experience of dropouts and completers, previous studies had found that 
completers were far more likely to persist in the labor force over time. This narrowing 
difference is probably the product of the faster growing economy and the increasing number of 
jobs available in the labor market during the follow-up period. 



Figure 6 - Quarterly Retrainee Labor Force Participation 
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New Hire trainees were also more likely than Controls to persist in the labor force after training 
relative to the Controls. In the fourth quarter after training 89.2% of New Hire completers and 
88.8% of New Hire dropouts were still in the labor force compared to 84.3% for Controls. Again 
the gap between trainees and Controls was smaller immediately after training, about 2%, but 
grew over the eight quarters to over 5% with completers and over 7% with dropouts. The 
surprising finding here is that dropouts were slightly more likely to persist in the labor force after 
training. It may well be that dropouts are leaving training because they have other valuable 
options, and that is showing up as slightly higher labor force attachment. 
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Figure 7 - Quarterly New Hire Labor Force Participation 
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Another measure of attachment to the labor force is a byproduct of our earnings analysis. To 
compare earnings before and after training we defined a subgroup of trainees who were in the 
labor force in the 4 th quarter before, and the 4 th and 8 th quarters after training. Applying this 
criterion we found that about 77% of all trainee groups — Retrainees and New Hires, completers 
and dropouts — persisted in the labor market for the period we studied. On the other hand, only 
70. 1% of Controls remained in the labor market for the entire period. Thus, the trainees are 
about 10% more likely to be attached to the labor force than the Controls. 



Figure 8 - Labor Force Attachment 
(Found in 4th Quarter Before and 4th & 
8th Quarter After Training) 
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The most striking feature of this analysis is the high level of labor force attachment found in all 
groups. Approximately 90% of all trainee groups and nearly 85% of the Control Group were 
still in the California labor force in the fourth quarter after training. This compares with our 
previous study which found, 94.1% of Retrainee completers and only 82.0% of Retrainee 
dropouts, 89.7% of New Hire completers and 68.8% of New Hire dropouts compared to 79.6% 
of Controls in the labor market the year after training. The labor force attachment of both groups 
of dropouts is much higher in this study than in the earlier study (Moore, Blake, Phillips, 1995). 

• Change In Earnings 



The change in annual earnings is measured from the year before training to the first year after 
training and then to the second year after training. As mentioned, we restricted this analysis to 
trainees who could reasonably be assumed to have been in California and available for work 
during the time period studied. This subgroup consisted of those trainees and Controls who were 
in the Ul-covered California labor market in the fourth quarter before training and fourth quarter 
and eighth quarters after training. We then compared the annual earnings and UI claims of 
Retrainees and New Hire completers and dropouts to that of the Controls. 



S Retrainees: Change in Earnings 



Table 2 



Group 


Change Year Before 
to Year 1 After 


Change Year Before 
to Year 2 After 




$ Change 


% Change 


$ Change 


% Change 


Retrainee Completers 


$2,625 


10.9% 


$3,625 


15.0% 


Retrainee Dropouts 


$2,390 


9.7% 


$3,649 


14.7% 


Controls 


$1,541 


6.0% 


$3,025 


11.8% 



Figure 9 - Percent Change in Annual Earnings - Retrainees 
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Figure 10 - Retrainees 
Annual Earnings 
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Retrainee completers and dropouts, as well as the Controls showed increased earnings for both 
years after training, as Table 2 indicates. This reflects the overall prosperity in the economy and 
generally rising earnings of California workers. Completers gained slightly over $2,600 in the 
year after training and dropouts gained about $2,400 compared to only about $1,500 for 
Controls. By the end of the second year of training, the earnings gaps narrowed substantially, 
while second year gains for completers and dropouts were both about $3,600 more than the year 
before training, and the gains for Controls rose to slightly more than $3,000, for a second-year 
difference between the trainees and Controls of $600. 

A variety of studies estimate the impact of employer-provided training on trainees earnings at 
between 5 % and 12% (see for example Erickson, et. al., 1998 or Lillard and Tan, 1986). The 
increases we observe fit with these earlier findings. The simple change in earnings from year 
before to year after training represents a 10.9% increase for Retrainee completers, a 9.7% 
increase for dropouts, against a 6% increase for Controls. Using the Controls as benchmark for 
what would have happened to completers without training, the impact of training was 4.9%, an 
effect similar to the lower estimates of earlier studies of employer provided training. 

Relative to the early study of ETP, the current findings show similar patterns of earnings gains 
for the Retrainees relative to the Control Group but different patterns between Retrainee 
completers and dropouts. The reader is cautioned that the previous study is not quite comparable 
to the current analysis. In the previous study earnings and UI claims were analyzed only for the 
subgroup found in the 4 th quarter before and the 4 th quarter after training (rather than the -4, +4, 
+8 quarters in the current analysis). 
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Retrainee completers had larger gains in earnings compared to controls in this analysis than in 
the previous ETP study. Current Retrainee completers gained $1,100 more in earnings than the 
Controls, between the year before and the year after training while the differential gain was $800 
in the previous study. (Retrainee completers gained $300 while Controls lost $500). 

A major difference between the two studies is the experience of the Retrainee dropouts. The 
previous study, done in the midst of a recession, showed that Retrainee completers had a small 
gain of just $300, while dropouts had a severe decline of $2,600. Perhaps now, in a prosperous 
economy, dropouts have better options than they had during slower economic times so that the 
gap between completers and dropouts is smaller. Research also suggests that current projects 
that generate a larger number of dropouts are often those where companies become too busy 
meeting customer demand to complete training (Ong, et. al. 1999). Thus, one reason for the 
strong performance of dropouts may be that many are found in companies who are very busy and 
are working overtime or receiving pay increases due to their companies’ success. An other 
interpretation could be that training is having a positive effect on dropout’s earnings. ETP does 
not enroll trainees until they complete at least 20% of the training. Hence each dropout received 
at least 20% of the training and perhaps much more. Indeed some trainees would have received 
all the training and not completed the required 90 day retention period. Hence some of the 
increase in dropout earning may be due to training. 



One additional intriguing piece of evidence is that Retrainee dropouts earned about $600 more 
than completers did in the year before training. In past studies, dropouts tended to earn 
substantially less than completers did before training. This may indicate dropouts are actually 
coming from more successful companies where earnings are higher. 

The actual amount the Retrainees earned is also revealing. As Figure 9 shows, Retrainee 
completers and dropouts both earned around $24,000 in the year before training, with dropouts 
earning a little more. Both groups earned less than the Controls, who averaged around $26,000. 
The higher earnings of the Control group may reflect the fact that this randomly selected group 
of workers from similar industries includes managers, engineers, and other well-paid workers 
while ETP projects are designed to serve frontline workers who tend to be hourly employees 
with less than a college education. 
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Figure 1 1 - Retrainee 
Quarterly Earnings 




A second dramatic difference, between the earlier (1991-92) Retrainees and the current 
Retrainees, is the difference in the amount of earnings. In the earlier study both completers and 
dropouts had earnings of almost $30,000 before training. The current group averages around 
$24,000, a decline of $6,000. This shift is probably due to several factors. Two prominent 
factors are changes in the industry mix and changes in the types of companies served by ETP. 
Through its large contracts with training agencies that train in areas such as office automation, 
ETP is reaching many more lower-skilled employees from smaller companies who pay less. 

Finally, we examine the pattern of earnings changes over the twelve quarters studied for all the 
groups as illustrated in Figure 10. As the graph indicates, all three groups had steadily rising 
earnings over the period studied. All three of the groups ended up earning about $1,000 more 
per quarter in the 8 th quarter after training than they did in the 4 th quarter before training, with 
training completers having slightly larger gains. 

S New Hires: Change in Earnings 

Patterns for New Hires were substantially different than those for Retrainees. Recall that New 
Hires differ from Retrainees in that New Hires do not have jobs when they begin training; this 
factor seems to affect the earnings outcomes. Both New Flire completers and dropouts as well as 
Controls showed increased earnings for both years after training as Table 3 below indicates. 
Completers gained slightly over $2,700 in the year after training, while, surprisingly, dropouts 
had slightly larger gains of over $2,900. These results may indicate that in a healthy economy 
New Hire trainees drop out of training to take other well-paid job opportunities, not available in 
less prosperous times. We also conjecture that in a prosperous economy the trainees with the 
fewest opportunities persist through to complete ETP New Hire training. Controls had much 
smaller gains than either trainee group, only about $1,500. By the end of the second year of 
training the gaps narrowed substantially. Completers and dropouts both earned more, but now 
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completers show slightly larger increases than dropouts, $3,660 compared to $3,537, while gains 
for Controls rose to slightly more than $3,000. 
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Table 3 



Group 


Change Year Before 
to Year 1 After 


Change Year Before 
to Year 2 After 




$ Change 


% Change 


$ Change 


% Change 


New Hire Completers 


$2,733 


1 1 .2% 


$3,660 


15.0% 


New Hire Dropouts 


$2,954 


1 1 .6% 


$3,537 


13.9% 


Controls 


$1,541 


6.0% 


$3,025 


1 1 .8% 



Figure 12 - New Hires Percent Change in Annual Earnings - 

Retrainees 
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In summary, the New Hire completers’ earnings rose $1,200 more than the Controls’ earnings 
from the year before to year after training, and rose $600 more than the Controls two years after 
training. New Hire dropouts’ earnings rose $1,400 more than the Controls’ earnings from the 
year for to after, and $500 more than the Controls’ earnings two years after training. 

The current pattern oiThe New Hire completers’ earnings relative to Controls’ earnings is similar 
to the previous one but the differences are not as large. In the previous study, New Hire 
completers gained $3,000 from the year before to the year after training compared to the 
Controls; in the current analysis the difference is $1,200. Though the current change in earnings 
for the New Hire completers is smaller than it was in the previous ETP study, it compares very 
favorably with the results of other studies of displaced workers reviewed earlier. 
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Figure 13 - New Hires 
Annual Earnings 
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The New Hire dropouts had a completely different experience than that reported in the earlier 
study. In the current study, the New Hire dropouts gained $1,400 from the year before to year 
after training, compared to a loss of $2,300 in the previous study. To the extent that the 
difference between the earnings change for a New Hire completer relative to a New Hire dropout 
approximates the value of completing training, it clearly shows that it is worthwhile to complete 
training in a recession, while the value is much less in an economic expansion. The 1991-92 
New Hire completer had earnings gains of over $2,500 while New Hire dropouts had earnings 
declines of $2,300 for a difference of $4,800. In the earlier, more difficult economic 
environment the value of completing New Hire training was substantial, while in the recent, 
more prosperous times the value of completing was small, ($100 in annual earnings at the end of 
the second year). This small difference was probably due to the many other good alternatives 
offered by the labor market. 

Another interesting pattern is that, unlike the Retrainees who earned less than similar trainees in 
the 1991-92 cohort, these New Hires earned substantially more than the 1991-92 New Hires both 
before and after training. The current New Hire completers earned about $24,400 before training 
and $27,130 after training, putting them roughly on a par with the current Retrainees. In 1991- 
92, New Hire completers earned only $14,200 before training and about $16,900 after, a 
difference of $10,000. These differences are principally due to the fact that current New Hires 
were unemployed much less both before and after training. 

Another dramatically different pattern is that in the past study New Hire Completers earned 
almost $3,000 more than dropouts before training. In this current cohort, the pattern is reversed 
and dropouts earn $1,000 more than completers before training. This lends support for the 
notion that dropouts may in fact, be somehow better qualified than completers in the current 
cohort, while the opposite was true in the past. 
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Looking at the quarter-by-quarter earning patterns of all three groups, we see that completers 
consistently earn less than either the Controls or the dropouts in the year before training. After 
training, all groups experience steady increases in earnings and the gap between the groups tends 
to narrow over the eight follow-up quarters. Completers, dropouts, and Controls all ended up 
earning about $1,000 more in the 8 th quarter after training than in the 4 th quarter before training, 
with completers making slightly larger gains than the other groups. 



Figure 14 - New Hires 
Quarterly Earnings 
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S Retrainees: Change in UI Claims 

Both Retrainee completers and dropouts were unemployed substantially more than the Controls 
before training, and then were unemployed substantially less than Controls after training. 
Specifically, Retrainee completers experienced a drop of 0.68 in average annual weeks ofUI 
claimed from the year before training at (1.84 weeks) to the first year after training (at 1. 16 
weeks). Retrainee dropouts were similarly unemployed 1.82 before training, falling to 1.16 
weeks after training. This compared with only a 0. 1 1-week drop for the Control Group, who 
started at 1.35 weeks in the year before training, and dropped to an average of 1.24 weeks in the 
second year after training. 
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Figure 15 - Retrainees 
Annual Weeks of Unemployment 
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The reduced levels of unemployment persisted into the second year, with average unemployment 
remaining virtually the same for both completers (1.15) and dropouts (1 12), while 
unemployment for the Controls rose slightly to 1.31 weeks. 

Again, the annual pattern here is essentially the same as the case of the 1991-92 cohort, except 
that the economic environments were very different In the earlier study, both Retrainee 
completers and dropouts had increases in the weeks unemployed after training. That pattern was 
due to the recession, which was worsening in California at that time (unemployment rose from 
5.5% to 7.7% during the study period). In the current study the economy and jobs were 
expanding and unemployment was falling, and the before-to-after weeks of unemployment of the 
Retrainee completers fell by about .6 weeks relative to the Controls (a .68 week decline for 
Retrainee completers relative to a . 1 1 for Controls). In the previous study, the weeks of 
unemployment were rising for both groups but the weeks of unemployment for the Retrainee 
completers rose .8 weeks less than for the Controls (a 1.6-week rise for Retrainee completers 
versus a 2 4-week rise for Controls). 

The data quarter-by-quarter unemployment data in the Figure 14 below shows a more divergent 
pattern Unemployment for Retrainees, (both completers and dropouts) rose prior to training 
while unemployment for the Controls fell. After training the level of unemployment was lower 
and largely steady for Retrainees, while it varies substantially quarter-by-quarter for the Controls 
and is higher than either group of Retrainees. 
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Figure 16 - Retrainees 
Quarterly Ul Weeks 
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UI payment data amounts mirror the pattern of weeks UI was claimed. Retrainees begin in the 
year before training by collecting more in UI payments, about $200, than Controls about $140. 
In the years after training both Retrainee completers and dropouts collect less in payments. 
Payments to completers fall $71 on average and dropouts $8 1 . This compares to a decline of 
only $9 for Controls. It is interesting to note that dropouts have slightly larger declines than 
completers do. Perhaps this indicates, as noted before, that dropouts failed to complete training 
not because they lost their job but because their companies got particularly busy, or they 
voluntarily changed to another job. 
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Figure 17 - Retrainees 
Annual Unemployment Payments 
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The quarter by quarter analysis shows the now familiar pattern of payment rising for Retrainees 
before training, falling and then remaining mostly constant. Conversely the Control Group has a 
brief decline after the training period and then varies quarter-to-quarter. 



Figure 1 8 - Retrainees 
Quarterly Ul Payments 
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S New Hires: Changes in UI Claims 

New Hires follow a similar pattern to the Retrainees. New Hire completers are unemployed 
about 1.93 weeks in the year before training. Unemployment falls to 1.09 weeks in the year after 
training and remains near there in year two. New Hire dropouts are unemployed slightly less 
before training. Their annual unemployment also falls to about 1.15 weeks in the year after 
training and remains there in the second year. In contrast, the Controls are unemployed less the 
year before training (1.35 weeks). They have only a small decline in the after-training period 
and end up unemployed more than the New Hires the years after training. From the year before 
training to two years after training, the New Hire completers unemployment drops by almost one 
week and dropouts’ unemployment falls about two-thirds of a week, while the Controls remain 
about the same. 

In contrast, the 1991-92 cohort of New Hires had much higher levels of unemployment before 
training, about 10 weeks. After training, unemployment fell by 5 weeks for completers and 
remained about the same for dropouts. 
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Figure 19 - New Hires 
Annual Weeks of Unemployment 
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An examination of the quarter-by-quarter trends for the current cohort of New Hires shows very 
different patterns for the New Hires and the Controls. As expected, the New Hires were 
experiencing a rapid rise in unemployment prior to training. In comparison, unemployment for 
the Controls was declining. This pattern makes sense given that trainees must be unemployed 
when they enter New Hire training. After training, the level of unemployment falls for both New 
Hire completers and dropouts, and remains slightly below the level of the Control Group. 
Completers are consistently unemployed slightly less than dropouts are. 



Figure 20 - New Hires 
Ul Weeks By Quarter 
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New Hire trainees’ Ui payments reflect their weeks of unemployment experience. UI Payments, 
from the year before to the year after training, fell more for the completers ($82) than the 
dropouts ($59). This compared favorably with Controls who only had declines of $9. The low 
levels of UI payment persisted through the second follow-up year for the New Hires while the 
Controls edged upward. 
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Figure 21 - New Hires 
Annual Unemployment Payments 
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The quarter-by-quarter analysis shows that after rising steeply above the level of the Controls 
before training UI payments fell to about the same level as Controls after training. Payments to 
completers remain below dropouts for almost all of the follow-up period, and then converged in 
the last two quarters. This pattern indicates that the benefits of completing training are strongest 
in the year or so after training. 

Error! Not a valid link. 
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• Employment Stability 

S Overview 

Part of ETP’s mission is to create “secure employment” for California workers. To see if ETP is 
achieving this goal we measured the employment stability of the workers in our two-year cohort 
of ETP trainees found in the year before and first and second year after training. For both 
Retrainees and New Hires, we wanted to determine whether they had more or less stable 
employment after training. To do this we constructed an “employment instability index.” The 
index combined data on unemployment and whether or not a trainee remained within the same 
industry. Index scores for ETP trainees, both dropouts and completers, were compared with 
scores from similar workers in the control group to estimate the impact of ETP training. 

S Method 

An employment instability index was constructed in the following way: 

>- The average number of weeks of unemployment in a given quarter in the year 
before and after training was computed for each of the five groups (Retrainee 
completers and dropouts, New Hire completers and dropouts, and controls). 

> The average number of industry changes, as reflected by a changing SIC for 
the primary employer, was computed for each of the five groups. 

>■ A ratio was computed for each ETP group, dividing each ETP value by the 
corresponding Control value (e.g., Retrainee Complete Average Weeks of 
Unemployment divided by Control Average Weeks of Unemployment, for 
corresponding quarters). 

' s> The mean ratio was computed (add the unemployment and industry change 
ratios, divide by two) and this is the “quarterly instability index.” If the index 
equals one, then the ETP group is the same in stability as the Control group. 

If the index exceeds one, then the ETP group is less stable, and if the index is 
below one, then the ETP group is more stable. 

S Results 

The graph below shows that Retrainee completers had more stable employment after training 
than either the control group or the Retrainee dropouts. Similarly New Hire completers had 
more stable employment than either the control group or New Hire dropouts. 

As the graph below indicates, in the year prior to training, we see instability increasing for all 
groups. It increases most dramatically for New Hires, which makes sense considering that they 
must be unemployed to eligible for the program. However if we look at the entire year before 
training, the Retrainee dropouts are indistinguishable from the control group (Average index, 
1.01), but the Retrainee completers are more stable (Average index, .90). The New Hire 
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dropouts had much higher instability than the controls (Average index, 1.30). Interestingly, the 
New Hire completers were substantially higher than the controls and the dropouts (Average 
index, 1.63). 

Looking at index scores in the year following the training, the Retrainee completers are 
significantly more stable (Average index, 73) than both their pretraining values and the controls. 
The Retrainee dropouts on the other hand, have only a small improvement in stability and 
remained close to the control group (Average index, 95%). The New Hire dropouts show a very 
slight increase in stability and remain substantially above the controls (Average index, 1.27). 
New Hire completers show a dramatic increase in stability going from an index of 1.63 before 
training to 1 .00 after training. Indicating that that their experience is now the same as the control 
group. 

Looking at the quarter-to-quarter changes that underlie the annual changes, we see the instability 
was rising for all ETP trainees before training. It declines dramatically immediately after 
training and continues to decline for two more quarters, before beginning to rise. By fourth 
quarter after training Retrainee completers, Retrainee drops and New Hire completers have 
begun to converge, while New Hire drops remain far less stable than other trainees. 

Clearly, completing training has positive impact, particularly on those New Hires who had the 
least stable experience prior to training. The increased stability that comes from completing 
training appears to persist for at least the entire first year after training for both New Hire and 
Retrainee completers. Conversely those who dropped out of training saw little improvement in 
stability in the year after training. New Hire dropouts remained much worse off than the control 
group and Retrainee drops continued to have an experience very close to the control group. 
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Figure 23 - Employment Instability 
Year Before and After Training 
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Figure 24 - Employment Instability by Quarter 
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• Regression Analyses 
^ Methodology 

The regression analyses were designed to detect any systematic relationships between changes in 
trainee earnings and demographic variables (age, experience, education, gender, and so forth) or 
programmatic variables (size or type of business, type of training, type of training contractor, and 
so forth). Knowledge of any systematic positive or negative effects of these trainee or contractor 
characteristics on changes in trainee earnings could be helpful in developing more efficient or 
effective training policies. 

In our search for insightful relationships between earnings changes and the demographic and 
programmatic variables, we investigated several alternative formulations of these relationships. 
The constant aspect of our analysis was that Retrainees and the New Hires were always 
investigated in separate regression analyses because of the very different nature of the trainees 
and their circumstances. This allowed the relationships between earnings change and the 
demographic and program variables to be different in magnitude and significance for Retrainees 
versus the New Hires. Had we forced these two groups of trainees into a single equation, all of 
the quantitative relationships between earnings change and the other variables would have had 
the same magnitude and statistical significance. 

The demographic variables investigated included 
age, 

>■ education, 

>■ gender, 

>■ ethnicity, 

>■ marital status, 

whether the trainee was a veteran, and 
~>- whether the trainee was handicapped. 

The programmatic variables included 

2^ whether the trainee completed training, 

>• the type of training, 

>• the type of trainer, 

^ the hours spent in classrooms, in laboratories, and in SOST, 

>• the size of the business, 

2^ the type of industry, and 

2^ whether the training was related to an apprentice program, and 
>• whether the training was related to the GAIN program. 

Our regression model was designed to uncover any systematic relationship between these 
independent variables and the trainees’ before-to-after training change in earnings. 

We also measured the change in earnings two different ways. First, we used the dollar change 
from the year before training to the year after training. The second measure was the industry- 
adjusted change in earnings, which was the dollar change in earnings for the trainee minus the 
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average dollar change in earnings for the members of the Control Group in the same industry 
group. 

We used several types of regression equation formulations to search for these systematic 
relationships. For example, we investigated the nature of the relationship between the type of 
training contractor and earnings change both as additive effects within one regression equation 
and in separate regression equations that allowed different effects between type of contractor, 
earnings change, and all other variables. We also used regression analysis techniques designed 
to help detect significant relationships in cases where there are many potential relationships. 

One technique was a step-wise regression that effectively searched the relationship between 
earnings change and the other variable for the most strongly related variable, then repeated the 
process for the next most strongly related variable, and so forth. We applied other analytical 
techniques like this one in our search for related influential demographic and programmatic 
characteristics. Some of our analytical techniques involved dropping variables from the analysis 
that had severe data omission problems to determine whether the remaining variables had 
significant relationships to earnings changes. The full regression equations with the all of the 
variables specified appear in Appendix B. 

■S The Data 

We encountered severe data problems in our analysis. The data files for the New Hire trainees 
were so riddled with missing data that we could not use 80% of the New Hire records in the fully 
specified regression analysis. That is, out of 4,336 trainee records, we had to delete 3,462 from 
this analysis due to missing data, using only 874 records in our search for statistically significant 
relationships. The Retrainee records fared only somewhat better in the final analysis, with only 
45% of the records discarded because of missing data (23,837 out of 53,121). 

This sad record of dat* usability occurred even after a major data cleanup effort in which ETP 
personnel searched old files for missing and incorrect data entries on trainee records (some 
records were archived and thus not readily available). In one critical data field, the employer ID 
tax code which allows the linkage between three critical databases: 1) the individual earnings 
and UI claims records, 2) the company and contract characteristics, and 3) the individual 
trainee — we corrected well over 4,000 incorrect and missing entries. In another less critical (but 
still important) data field we compared the Business Size codes listed in the Contract Database 
with those listed in the Trainee Files. We found disagreement between 10,818 codes out of a 
total of 37,127 records in the 94-95 file. The disagreements included 5,521 records where no 
business size was coded. In the 95-96 file we found disagreement between 4,719 cases in the 
contracts (out of 20,346 total records) with 2,626 of those showing no business size code. These 
are very large error and omission rates. 

While we were able to discover and enter or re-enter many correct codes with a great deal of 
time and effort, our ability to discover erroneous entries was limited to areas where two 
databases contained similar information allowing cross-checking. Most of the data fields did not 
allow cross-checking so undoubtedly many entry errors went uncorrected. Furthermore, there 
was no alternative source for most missing data, so those data were unavailable for the analysis. 
Unfortunately, the large error and omission rate affected the regression analysis results. 
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It is not clear why the data records are in such bad shape for the trainees in the current study 
period. In the a previous study of ETP involving the 1990-91 trainees, we were able to use 
89.7% of the Retrainee records (16,249 out of 18,1 18) and 93.5% of the New Hire records in the 
regression analysis. In the current study we were able to use only 55.9% of the Retrainee 
records, and only 20.5% of the New Hire records in the regression analyses (Moore, Blake & 
Phillips, 1995). 

^ The Findings 

Our findings were disappointing. Our specified regression equation could only explain 1% to 
8% of the total variation in earnings change among the trainees (this explanatory power is 
measured by the r-squared of the regression). While regression analyses of cross-sectional data 
like these typically have low explanatory power because of variations caused by factors not 
captured in the regression equation (earnings history, personal characteristics, quality of training 
and education, and so forth), the explanatory power of these regressions are very low even by 
cross-sectional standards. Regressions with r-squares in the 20% to 40% or more range would be 
much more significant in that they would contain some variables that explained some significant 
portion of the observed variation. 

The poor performance of these regression equations can probably by attributed to the data quality 
problems. When as much as 80% of the observations have to be discarded because of missing 
data, the tighter statistical distributions associated with the law of large numbers and other 
statistical tendencies have little opportunity to take effect. The results are wide variations among 
the remaining data and low statistical significance for means and other statistical measures. The 
types of errors found also introduced wide variations in the data. For example, in most cases the 
trainees in a single company would code company size at least three different sizes out of the 
four available categories (1-50 employees, 51-100, 101-250, or more than 251). Another 
variation was introduced by changing the coding scheme for education level during the study 
period. For part of the period there was no code for having completed some high school (more 
than eighth grade but not a high school graduate). These kinds of variation-inducing errors and 
omissions reduce the explanatory power of any regression equation and decrease the statistical 
significance of any examined variable. The results of this analysis are a case in point. 

We did find a statistically significant result in one of our regression equations. This result 
derived from a statistically significant constant term in one of our Retrainee regression equations 
on industry-adjusted earnings change. Retrainees experienced annual earnings growth of $957 
more than the average in their industry from before to after training. Notably, it did not matter 
whether the particular trainee completed training, because the regression coefficient 
corresponding to completing training was statistically insignificant in this and other regression 
equations. This means that Retrainee dropouts experienced the same earnings boost relative to 
their industry cohorts that the Retrainee completers did. This result is consistent with the results 
reported earlier in this study. 

We did find three statistically significant coefficients in one of the reduced regression 
formulations for New Hires. Unfortunately two of these coefficients were associated with the 
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educational variable which had been variously coded, and the other was associated with business 
size which also had been unreliably codes. Given that over 75% of the observations had to be 
deleted because of missing data even for this reduced equation, we have no confidence that these 
results provide any insight into New Hire training relationships. 
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The Economic Impact of ETP Training 



The impact of ETP training on the California economy is the difference between what did occur 
in the California economy with ETP training programs in place and what would have occurred in 
their absence. ETP's impact on the California economy is estimated at over $400 million in the 
first year after training. This estimate is based on the impact training provided by ETP in 1994- 
95 and 1995-96. ETP funds invested to achieve this estimated economic return equalled $73.3 
million which is the total amount earned in contracts which completed in those two years. 

The economic benefits of these two years of training derive from the outcomes of training. 
Below, we identify several training outcomes and their economic benefits they generate. For the 
reader’s convenience we have summarized the estimated magnitude of those benefits for the 
California economy at the outset. The several sections that follow explain why those benefits are 
expected and carefully detail their empirical derivation. In each case we have been careful to 
include only net benefits to California, and to exclude a benefit to one California worker or 
company that appears to come at the expense of another California worker or company. 



• The Training Outcomes and Their Economic Benefits 



The benefits of training for the trainees can lead to economic benefits for the California 
economy. For example, training that increases employment stability will reduce UI claims by 
workers, and training that increases productivity will increase workers’ earnings and may spawn 
other economic activity in the state. The three types of ETP training outcomes with their main 
types of economic benefits are listed below. The benefits are estimated for the first year after 
training and are rounded to the nearest million dollars. 



Training Outcomes 

1 . Increased employment stability for trainees 

S Unemployment Insurance fund savings 

2. Increased productivity of trainees 

S Increased trainee earnings 
S Increased sales by other California businesses 

3. Saving California jobs 

S Unemployment Insurance fund savings 
■S Prevented temporary earnings losses by trainees 
S Prevented losses by other California businesses 



Estimated Benefits 



$2 million 

$33 million 
$17 million 

$61 million 
$167 million 
$134 million 



The outcomes the ETP training and their benefits for the California economy are summarized in 
Table 4. We estimate that ETP training during the two-year study period had an impact of 
approximately $414 million to the California economy in the first year after the training took 
place. The largest component of these benefits resulted from the ETP training outcome of saving 
jobs that were threatened by out-of-state competitors. This training outcome produced over $360 
million in benefits and entailed UI Fund saving, trainee earnings savings, and indirect (or 
multiplier) impacts, with the largest component being the impact on earnings saved in basic 
industries. The second largest benefit was associated with the productivity increases resulting 
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from ETP training. These benefits totaled nearly $50 million, with the direct impact of 
productivity increases on workers at almost $33 million and the indirect impact equal to about 
half of that, at $16.8 million. The smallest training outcome benefit was derived from increased 
employment stability which saved UI Funds over $2 million in the first year after training. 






Table 4 



I mm 



Outcome / Benefit 


UI Fund Savings 


Earnings Impact 


Indirect Effects 


Total 


Employment Stability 


$2,030,000 


NC 


NC 


$2,030,000 


Productivity Increase 


NE 


$32,815,000 


$16,766,000 


$49,581,000 


California Jobs Saved 


$61,115,000 


$167,305,000 


$133,845,000 


$362,265,000 


Total 


$63,145,000 


$200,120,000 


$150,611,000 


$413,876,000 



NC: Not calculated separately. Increased employment stability would lead to higher earnings for trainees 
due to less unemployment during the first year after training. We did not net this number out of the 
productivity increase estimated for the Retrainees , so any effect of increased employment stability for 
Retrainees is included in the productivity increase and indirect effects estimates. Any effects for New Hires 
are arguable because the training qualifies them for more stable jobs that are already available. 

NE: Not Estimated. Since UI taxes apply only to the first $7 ,000 of calendar year earnings and the 
productivity increase boosts earnings that already average around $25,000 annually, any effect of the 
productivity increase on increasing UI funds would be both small and difficult to estimate. 



Another way to read Table 4 is in terms of the type of benefit from ETP training. For example, 
UI Fund savings resulted from both increased employment stability and from saved jobs, and 
total over $63,000,000. ETP’s impact on earnings totals over $200,000,000 in the first year after 
training and is derived from both increased productivity and saving basic jobs. The indirect 
effects of ETP training on the California economy in the first year after training added up to over 
$150,000,000 and resulted mainly from saving basic jobs and secondarily, from increasing 
productivity in basic jobs. These estimated indirect effects may be subject to some cynicism 
largely because indirect or multiplier effects are often claimed with little logical justification and 
even less empirical basis for the multipliers used. However, the multiplier effects claimed here 
have been carefully justified and meticulously tracked and estimated. Further evidence of the 
existence of these effects is found in the section on the impact of ETP training on companies. In 
that section, companies with ETP training programs were shown to have grown faster than 
similar-sized companies in their industry. It is logical that as these ETP contracting companies 
which grow faster, also order more from their California suppliers, which then generate the 
indirect or multiplier effects. 



Next we show the analyses that lead to values reported above. 

• Outcome 1: Increased Employment Stability — Approximately $2 million 

A major goal of ETP is to provide workers with “secure employment.” Achievement of this goal 
can be measured by the reduction in their unemployment claims and by the lower UI payments 
distributed to them. As shown in Figures 13 and 17 in the previous section, ETP New Hires and 
Retrainees both experienced lower rates of unemployment (as measured by average UI weeks 
claimed) after training, relative to before. The improving economy during the study period 
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produced lower unemployment for the Control Group as well, but trainee claims rate fell 
significantly relative to the Controls. UI fund savings result from the lower U1 claims associated 
with lower post-training unemployment rates for the trainees. 

We estimated the reduced UI payments made to trainees as a result of ETP training by tracking 
the unemployment and UI claim experience of the Retrainee and New Hire completers compared 
to that of the Control Group. Because we were interested in estimating the total UI payments 
savings associated with ETP training, we tracked the experience of aU trainees relative to all 
Controls. (That is, we did not restrict the analysis to a subset of the trainees who were in the 
labor force in certain quarters before or after training, nor to Controls who were in the labor force 
in similar quarters.) Our analytical procedure involved first determining what percentage of 
trainees would have experienced unemployment had they not been trained. Then we compared 
that percentage to the completers’ actual post-training unemployment, and applied the trainees’ 
UI payment rate to the estimated difference in unemployment rates for the trainees and the 
Controls. 

We used the experience of the Control Group 4 to estimate the percentage of Retrainees and New 
Hire completers who would have experienced unemployment. However, we could not simply 
apply the unemployment percentages for the Control Group to the trainees because ETP 
contracts have selective requirements that imply the trainees could be different than the average 
California workers. ETP contracts require that the New Hire Trainees be out-of-work, and that 
Retrainee contracts involve workers who are threatened with displacement, or are transitioning to 
a high-performance workplace, or are in a company that is diversifying its product line. Because 
of these selective contract requirements, the ETP trainees are not expected to have the same 
unemployment experience as a random sample of the California workers (the Control Group). 
What can be expected, though, is that the ETP trainees experience the same trend in 
unemployment as the Control Group, because that trend is produced by the general conditions 
and dynamics in the California economy. In short, we assumed that, without training, the New 
Hires and Retrainee completers would have experienced the same trend in unemployment that 
the Control Group did. Tables 5 and 6 show the calculations that resulted in the estimates of this 
source of UI fund savings by ETP training for Retrainees and New Hires respectively. 



4 The Control Group experiences used throughout this analysis are those for the industry-adjusted Control Group. 
That is, the experience of the Control Group in UI claims or earnings or other variables was derived by weighting 
the Control Group according to the industrial composition of the trainees. This was done by determining the 
experience of the 25 industry-based subgroups of the Control Group and then weighting those industry-based 
subgroup data by the proportion of trainees that appeared in the same industry subgroup. This procedure yields a 
Control Group has the same industrial composition as the trainees. 
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. Table 5: 

Retrainee Completers’ Estimated Ul Savings Attributable to Training 



j 
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Item 


Year Before 


Year After 


Second Year After 


Retrainees: Average percentage 
making quarterly claims 


6.82% 


5.29% 


4.42% 


Controls: Average percentage 
making quarterly claims 


7.15% 


7.04% 


6.43% 


Normalization ratio* 


.954 


.954 


.954 


Retrainees: Adjusted percentage 
making quarterly claims 


6.82% 


6.71% 


6.13% 


Retrainees: Adjusted claim rate as 
a percent of actual claim rate 


100.00% 


126.96% 


138.79% 


Retrainees: Actual UI claims total 
payment 


$9,303,400 


$7,338,217 


$6,020,201 


Retrainees: Estimated UI 
payments without training 


$9,303,400 


$9,316,600 


$8,355,437 


Retrainees: Estimated UI savings 
due to training 




$1,978,383 


$2,335,236 



*Note that the normalization ratio is the Retrainees' year-before claims divided by the Controls' year-before claims. 
Applying this ratio to each of the Controls ’ claims rate produces the expected claims rate for the Retrainees. The 
Retrainees’ year-before expected claims rate is exactly what it actually was because of the definition of the 
■ normalization ratio. The Retrainees’ year-after expected claims rate is the Controls ' rate times the normalization 
ratio, which is what the Retrainees would have experienced if their claims rate had followed the same trend as the 
Controls ’ did. 



Table 5 details the procedure for estimating the UI savings due to training for the Retrainee 
completers. The first two rows of the Table report the average quarterly percentage of 
Retrainees and Controls who made UI claims during the year before and the two years after 
training. The “normalization ratio” in the third row is a ratio of the average quarterly percentage 
of retrainees making UI claims to that of the Control Group for the year before training. This is 
the ratio that was applied to the Control Group “after-training” claims rate to estimate the 
adjusted claims rate for the Retrainee completers. The result is the fourth row in the Table which 
shows the estimated quarterly average of UI claims that the Retrainees would have made had 
they followed the same trend in UI claims as the Control Group did. The Retrainees’ after- 
training adjusted claims rate was then divided by their actual claims rate to obtain the adjusted 
rate as a percent of the actual rate. This percentage was applied to the actual UI payments to the 
Retrainee completers to obtain the estimated total UI payments to the Retrainees had they not 
been trained. The Retrainees’ actual UI payments were then subtracted from the adjusted 
payments to obtain the estimated UI savings due to training for the Retrainee completers. The 
same procedure was followed for the New Hire completers in Table 6. 
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• New; Hire Compfet 


s|;g:^gfi^f.:T a ble.6 . -• 

ers’ Estimated UI Savings Attributable 1 to Training f 


Item 


Year Before 


Year After 


Second Year After 


New Hires: Avg percentage 
quarterly claims 


6.52% 


5.66% 


4.26% 


Controls: Avg percentage 
making quarterly claims 


7.15% 


7.04% 


6.43% 


Normalization Ratio 


.912 


.912 


.912 


New Hires: Adjusted percentage 
making quarterly claims 


6.52% 


6.42% 


5.86% 


New Hires: Adjusted claim rate 
as a percent of actual claim rate 


100.00% 


113.42% 


137.74% 


New Hires: Actual UI claims 
total payment 


$414,169 


$384,504 


$280,262 


New Hires: Estimated UI without 
training 


$414,169 


$436,104 


$386,033 


New Hires: Estimated UI 
savings due to training 

^ C/J/J t Li SI ll Si t St St 11 1 / 1 St It l St AAA Smml / . 




$51,600 


$105,771 



* See the note on Table AAA for an explanation of how the normalization ratio is calculated. 



The total UI fund savings associated with the lower UI claims rates of the Retrainee completers 
and the New Hire completers is estimated to be about $2,030,000 in the first year after training 
for those trained in the study period. As indicated in the Tables above, we estimated another 
$2,335,000 in UI fund savings for the second year after training for these same trainees, but this 
sum is not included in our subsequent calculations since our estimates pertain only to impact of 
ETP training in the first year after training. 



• Outcome 2a: Productivity Increase and Its Impact on Earnings — Approximately $33 
million 

The productivity of ETP trainees may increase either because they produce more per hour 
worked or because they work more hours in a year. Any training program that increases the 
marketable skills of the participants creates additional economic capacity to produce. ETP 
programs may have an advantage over other training programs because of the placement 
requirement in ETP contracts. Other training programs create the potential to produce more; that 
potential is realized when the newly trained workers are placed. ETP contracts require 
placement as a condition of training, so when ETP training is completed the economic potential 
of the enhanced productivity is realized. Thus, the impact of ETP training is realized 
immediately in increased output for the California economy, due to the enhanced productivity of 
the trainees who complete the program. In addition, because trainees gain new skills, they may 
have the opportunity to work more hours by avoiding periods of unemployment associated with 
less marketable skills. 
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Direct measurements of changes in productivity are difficult and expensive. Direct measurement 
of productivity requires accurate physical output and input data for the periods before and after 
training. Such data are generally not available, and ETP projects are no exception. Even if those 
data were available, direct measurement of individual productivity is complicated because newly 
trained workers typically work with different equipment, different materials, and differently 
skilled people than they did before training. It is difficult to separate a single worker's change in 
productivity from the contribution made by the new equipment, new materials, and differently 
skilled workers. 

In this project, as in most studies, the change in earnings is used as an indicator of the change in 
worker productivity. Standard economic theory implies that workers are paid the value of their 
contribution to production (their marginal product). Accordingly, an increase in productivity 
should result in an increase in workers' earnings. The advantage of using earnings as an 
indicator of productivity is that earnings data are much more available than physical output data 
and earnings data are reported on an individual basis. Because uniform records of the physical 
change in production levels do not exist for ETP projects, we used the changes in earnings data 
as indicators of changes in productivity 5 . 

Worker productivity can be influenced by a variety of factors other than training These factors 
include: investments in capital equipment (new machinery, computers, or software), 
improvements in infrastructure (usually transportation, utilities, and communications systems), 
growth in technology, more efficient regulation, or a simple increase in the value that consumers 
place on the products produced. Thus, to estimate the increase in worker productivity attributable 
to ETP training, we need to control for the productivity increases due to these other factors. We 
used the pattern of earnings changes for the Control Group to adjust for the changes — other than 
ETP training — that could affect trainee productivity. 

The increase in productivity attributable to ETP is the difference between the trainees’ actual 
growth in earnings and the growth in earnings that they would have had without ETP training 
We used the earnings of the Control Group as an indicator of the average rate of productivity 
increases for workers who were not in ETP training 6 Again, since we were trying to capture the 
total impact of ETP on productivity, we used the experience of all of the trainees and all of the 
Control Group in these estimations. The results of this analysis are presented in Table 7. 



5 Some researchers argue that changes in wages underestimates the increase in productivity because some common 
pay practices (see for example Bishop, 1985). If this is correct then these estimates understate the real impact of 
ETP training. 

Note that this use of earnings of the Control Group would include the productivity effects of the average level of 
training going on in the California economy because that training would affect the average earnings in the labor 
force. 
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Table 7 .Y v * 






Item 


Year Before 


Year After 


Second Year After 


Retrainees: Total 
Earnings 


$999,135,817 


$1,060,014,095 


$1,051,697,931 


Percentage Based on 
Year Before 


100.00% 


106.09% 


105.26% 


Control Group Earnings 
Percentage Based on 
Year Before 


100.00% 


102.91% 


105.33% 


Retrainee’s Expected 
Earnings Based on 
Control Group Pattern 




$1,028,210,670 


$1,052,389,756 


Retrainees’ Excess 
Earnings Growth 




$31,803,426 


$(691,825) 


New Hires: Total 
Earnings 


$47,899,614 


$50,303,608 


$49,781,648 


Percentage Based on 
Year Before 


100.00% 


105.02% 


103.93% 


Control Group Earnings 
Percentage Based on 
Year Before 


100.00% 


102.91% 


105.33% 


New Hire’s Expected 
Earnings Based on 
Control Group Pattern 




$49,293,493 


$50,452,663 


New Hires' Excess 
Earnings Growth 




$1,010,115 


$(671,015) 



Table 7 shows the before-to-after earnings patterns for the Retrainee completers and the New 
Hire completers. These patterns are converted into percentages based on the earnings in the year 
before training. The next line shows the before-to-after earnings patterns for the Control Group, 
also in percentages based on the year before. We then multiplied the before-training earnings of 
the trainees by the Control Group’s before-to-after percentage income change to determine what 
the trainees would have earned had they followed the same earnings trend as the Control Group. 
Using the Control Group as an indicator of the expected earnings pattern for the trainees, had 
they not been trained, the difference between the expected pattern and the actual pattern in 
converted into dollars in the last line. Thus the estimated year-after training increase in 
productivity, attributable to ETP training, is approximately $32,815,000 for the Retrainee and 
New Hire completers combined. 

We included the estimated second-year-after productivity differential to illustrate a major finding 
of this analysis. Namely, that the effects of ETP training in a recovering economy appear to be 
substantially different than in a slowing economy. Table 7 shows that the productivity- 
enhancing effects of ETP training disappeared in the second year after training in the recovering 
economy. This result is quite different from the results of the earlier studies of ETP where 
productivity gains persisted over at least three years. Other studies have suggested that the 
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training-related productivity gains lasted about 12 years, and the earlier ETP study results were 
more or less consistent with that result (Lillard and Tan, 1986). Unfortunately, we did not have 
additional years of data for these trainees so we could not determine whether some of the 
productivity-enhancing effect of training returned during the third or fourth years after training. 
We have omitted any projection of the long-term effects of ETP training because of the rough 
equality between the trainees’ and Control Group’s earnings growth in the second year after 
training. Without subsequent years of earnings growth experience for trainees and the Control 
Group, we are unable to determine whether there is a long-term enhancement of productivity for 
trainees in an expanding economy with falling unemployment. 



• Outcome 2b: Increased Productivity and Its Multiplier Effects — Approximately $17 
million 

ETP training programs have the potential to have an economic impact beyond the direct effects 
on the program participants and the companies which employ them. These are indirect or 
“multiplier” effects, and they are transmitted from one company that is experiencing growth to 
their supplier companies through increased orders. Whether these indirect effects are realized 
depends on the nature of the companies in which the trainees are placed and the type of trainee. 
Specifically, for the potential indirect economic impact to be realized, the companies must be in 
basic industries and the trainees must be Retrainees . We explain the reasons for these conditions 
after a brief explanation of regional multipliers. 

In standard economic theory, regional multipliers are derived from the distinction between 
“basic” activities and “service sector” activities. The businesses that operate in these competitive 
national and international markets are the “economic base” of the region’s economy. According 
to the regional theory, these “basic” activities generate the jobs in the “service sector” of the 
local economy through orders from their local suppliers and through payroll expenditures that 
generate local household demands. The “services sector” includes all the businesses that are 
suppliers to locally- situated industry and households, or, put another way, businesses which 
respond to locally-generated demand for goods and services. Regional theory postulates that 
variations in the output of the basic industries spawn variations in their local orders and their 
payroll, which in turn cause variation in the demand for the service sector products. Regional 
multipliers show the relationship between a change in the output of a basic industry and the 
output of a region as a whole (basic plus service sector). Basic industries that have a greater 
proportion of local suppliers and high local payrolls relative to output have larger multipliers. 
Basic industries that have weaker linkages to the local economy have smaller multipliers. 

The service sector is typically identified with local suppliers of large firms, retail trade, local 
business services, most personal services, and local government activities. In contrast, 
businesses that supply demands generated outside the local economy are part of the economic 
base. The economic base encompasses all export-oriented activity and includes: the portion of 
local hotels, restaurants, entertainment and retail trade that services visitors; governmental 
activities funded by non-local sources; and interregional financial, insurance, transportation, and 
utility networks. 




Training That Makes a Difference 



48 



41 



The bottom line is that basic industries as those which compete with out-of-state businesses. 
Whether a particular retraining program has a multiplier effect in the California economy thus 
depends critically on the location of the competitors of that business. If that company’s 
competitors were located out-of-state then the increased sales for this company and its suppliers 
would come mainly at the expense of out-of-state firms and suppliers. This would produce a net 
gain in production and jobs for California. Similarly, if this company lost its competitive 
position, its lost sales would likely be to have been picked up by out-of-state companies and their 
suppliers, resulting in a loss of economic activity in California. On the other hand, if a 
company s competitors were other California businesses then this company’s and its suppliers’ 
gains or losses would produce offsetting effects in other California businesses, and yield 
essentially no net gain or loss for the California economy. For example, a company that 
manufactures computer printers competes with companies that are out-of-state or in other 
countries and thus the computer printer manufacturer is in a basic industry. Conversely, a 
restaurant does not compete with out-of-state restaurants, but with other local restaurants and is 
thus not in a basic industry. 

ETP training that enhances a firm’s competitive advantage produces an economic impact beyond 
the direct effects of ETP on the businesses and the workers. Because ETP retraining programs 
upgrade both the workers’ skills and their jobs, these ETP training strengthens the advantage of 
California businesses relative to their competitors. To the extent that California businesses with 
ETP retraining programs gain sales, or preserve sales they would otherwise have lost to out-of- 
state competitors, this means more production for them and increased sales and jobs for their in- 
state suppliers. The impact of the retraining programs on the California economy would be the 
increased production in the businesses with the retraining programs, plus the increased activity of 
their various suppliers. The aggregate of the increased production of their suppliers is the 
indirect effect,” of the direct impact on the businesses with retraining programs. This indirect 
economic impact is called a “ripple or multiplier effect” and it requires that there be no offsetting 
negative effect on other California suppliers. California’s economic environment and the 
conditions that lead to multiplier effects in the California economy are described below. 

California has the largest economy of any state in the nation and is a major trade center for the 
Pacific Rim. Recent advances in communications and information processing technologies, 
along with falling transportation costs, have produced keen competition in the markets for 
nationally and internationally traded commodities and services. Consequently, California 
businesses that produce, warehouse, transport or sell nationally and internationally traded goods 
and services face rigorous domestic and international competition. 

The simple classification of the firm's industry as either basic or service sector usually 
determines whether a particular training program will have multiplier effects. However, there 
are some cases in which retraining programs in the service sector could also have multiplier 
effects. This would occur in the case where a firm’s market consists of local customers but the 
firm competes with out-of-state producers who also supply this local market. This is the case of 
“import substitution,” when an increase in the firm’s output takes place at the expense of 
imported goods instead of locally-produced goods. If new local production replaces consumer or 
production goods and services that would have otherwise been imported from outside the region, 
then this new locally-oriented production generates additional local jobs. In short, this import - 
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substitution type of service activity can have multiplier effects. If all of the service sector 
training done by ETP were in businesses which competed with non-California suppliers, then the 
productivity enhancement in those industries would also be subject to a multiplier effect on local 
jobs. The multiplier effect accounts for the new economic activity created in the service sector 
as local suppliers respond to the higher demands from the firm and the community. 

The second requirement for ETP training to have indirect or multiplier benefits is that it involves 
Retrainees. This results from the fact that Retrainee programs involve upgrading jobs as well as 
workers’ skills. By upgrading jobs, businesses gain a competitive edge in the market and 
therefore will order more goods and services from their suppliers. Our assumption here is that 
the companies will increase all of their inputs more or less proportionally with the increased 
productivity of their retrained workers. This means that their orders from local suppliers would 
rise in proportion to their workers’ increased productivity (measured by earnings). An additional 
effect occurs as the Retrainees become more productive, earn more, and thus spend more in the 
local economy, generating additional economic activity in that way. 

In contrast, ETP New Hire training involves workers who are currently unemployed and require 
training to qualify for jobs that are already available. An important feature of New Hire training 
is the presumption that the job is available, and that if the New Hire trainee did not fill the job, 
someone else would. Thus, the consistent assumption for the New Hire trainee is that the 
training increases the productivity of the newly-trained worker, but does not increase the level of 
production in the host firm. This assumption — that no new or more productive jobs are created 
in training New Hires — limits the benefits of New Hire training to the increase in productivity 
experienced by the worker. That is, the benefit of New Hire training is only the increased 
productivity of the work force and the productivity of California businesses is not enhanced 
through New Hire training. This assumption is conservative but it is justified by the nature of 
most of the New Hire training projects. This conservative assumption is maintained throughout 
the series of estimates. Multiplier effects are assumed only for the retraining programs where job 
upgrades were present. 

This analysis suggests that ETP could be most effective in fostering economic growth in 
California by concentrating its retraining programs in companies basic industries or involved 
with import substitution. Retraining in these industries would increase output both indirectly 
(the multiplier effect) as well as directly (the productivity effect). Indeed, as of January 1, 1994, 
legislation required that all ETP retraining contracts involve workers in basic industries except 
those involving Special Employment Training (less than 10% of their budget). However, many 
of the contracts completed during this study period were begun before January 1, 1994, and were 
not subject to the basic industries requirement. 

The multipliers used in this study were developed by the California Department of Commerce, 
Office of Economic Research. We used the set of industry-specific, statewide job multipliers 
that include the effects of both additional industry and consumer spending in response to an 
increase in basic activity. To obtain an overall, average multiplier number, we weighted the 
industry multipliers by the percent of trainees in those basic industries. The weighted average 
multiplier that covered both industry and consumer spending was 1.8. This multiplier value 
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indicates that for 100 dollars of new activity in the basic sector, an additional 80 dollars of 
activity is created in the service sector. 

It is common to apply the multipliers to changes in basic activity to determine the effect of that 
change on total activity. We used a list of industries that have been determined to be basic 
industries in California and ETP contract information to determine the likely extent of retraining 
programs that are in basic industries or involved with import substitution. Our analysis of the 
ETP projects indicates that 65.9 percent of the Retrainee completers were in companies with 
potential multiplier effects 7 . 

Table 8 shows the calculation of the multiplier effect of ETP’s retraining programs on the 
California economy. The increase in Retrainee productivity estimated above ($3 1,803,000) was 
multiplied by 65.9% to obtain the increase in Retrainee productivity that occurred in basic 
industries ($20,958,000). The result was multiplied by the indirect effects part (.8) of the 
weighted multiplier of 1.8 to obtain the estimated multiplier effect. The indirect or multiplier- 
based benefits to the California economy from the Retrainees’ increased productivity were 
estimated at $16,766,000. This means that the enhanced productivity of the Retrainees who were 
employed in basic industries generated an additional $16,766,000 in business for suppliers to 
those basic industries and to businesses the Retrainees patronized. 



Indirect or MultiplierEffects of Retraining ^ . * \ ; 


Increased 

Productivity 


Percent of 
Retrainees in 
Basic Industries 


Increased 
Productivity in 
Basic Industries 


Weighted 
Multiplier for 
Basic Industries 


Indirect Impact of 
Retraining on 
California 


$31,803,000 


65.9% 


$20,958,000 


.8 (of 1.8) 


$16,766,000 



• Outcome 3a: Saving Jobs Protects UI Funds — Approximately $61 million 

One purpose of ETP training is to prevent unemployment among incumbent workers who are in 
jeopardy of being displaced because the company is not successfully meeting its out-of-state 
competition. Presumably, if the company does not meet its competition, it will lose market 
share, have to cut production, and lay-off workers as a result. Consequently, the threat of worker 
displacement is one of the bases for applying for ETP funding. Company contracts based on this 
rationale must show that retraining their workers will enhance their competitive position and 
reduce the threat of displacement to their workers. In these contracts, the ETP retraining 
programs directly enhance the competitiveness of California businesses by increasing both the 
skills of the workers and the skill requirements of those workers' jobs. Had the ETP programs 
not existed, the retraining and job upgrading presumably would not have occurred and these 
companies would have lost sales and jobs. If these jobs were in basic industries, these jobs could 
have been lost to the California economy, at least temporarily. A temporary loss of these jobs to 
California could occur while the current companies restructure or, if those companies retrench, 
until other California businesses recognize the market opportunity and reestablish the lost jobs. 



See California Economic Growth . Center for Continuing Study of the California Economy, Palo Alto, Ca., 1999, 
pp. A1-A5, for the list of basic industries. 
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On the other hand, if the threatened jobs were not in a basic industry, the some other local 
(California) business would have gained the sales lost by the threatened company and there 
would have been no loss to the California economy. The benefits to the California economy of 
preventing job loss is limited to those retraining programs that are in basic industry companies. 

The economic benefits of reducing the threat of lay-off of the Retrainees are two-fold. First, UI 
funds are saved because laid-off workers would have drawn on their UI benefits while they 
sought new jobs. We estimated the UI fund savings from this source from the number of 
Retrainees threatened with displacement, the estimated number of weeks they would have been 
unemployed, and their UI weekly payment rate. The second type of economic benefit of 
preventing lay-off is that whatever earnings the retrainees that would have been lost had they 
been displaced are not lost. We estimated the “saved” earnings of retrainees from the number of 
Retrainees threatened with displacement, the estimated number of weeks they would have been 
displaced, and their pre-training earnings. 

The percentage of Retrainees who were threatened by displacement during the study period is 
reported in Table 9. Our estimate of the number of Retrainees threatened with unemployment 
came from an examination of ETP retraining contracts. Under the Panel’s current legislation and 
regulations most ETP retraining contracts must satisfy one of following three requirements: 



^ the workers must be threatened with displacement if retraining is not made available, 
S the company must be transitioning to a high-performance workplace, or 
^ the company must be diversifying its product line. 

This legislative requirement pertains to contracts initiated after January 1, 1994. Before then, all 
ETP retraining contracts required that the workers must be threatened with displacement. Some 
of retraining contracts :n the study period were initiated before January 1 , 1994, and we assumed 
all Retrainees covered by those contracts were in jeopardy of lay-off. We identified the criteria 
for those contracts initiated after January 1 , 1994, and included only those Retrainees in contracts 
that were rationalized by a threat of worker displacement. Table 9 shows the breakdown of 
Retrainee completers who were threatened by displacement in the two contract years that were 
studied. 












„ gjiirabte 9::; 

Percent of Retrainee Completers Threatened by Displacement 





1 994-95 Contracts 


1995-96 Contracts 


Both Years 


Percent threatened by 
displacement 


95.3% 


55.9% 


81 .0% 


Percent not threatened by 
displacement 


4.7% 


44.1% 


19.0% 


Total 


100.0% 


100.0% 


100.0% 



Our estimate of the average number of weeks experienced, newly-unemployed workers would be 
unemployed in the following year came from the Control Groups. We identified those workers 
in the 1993, 1994, and 1995 Control Groups who made an unemployment claim in their selection 
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quarter, but not in the previous quarter, and then tabulated their average weeks of UI claims 
during that year (the selection quarter plus the next three quarters). Because this number of UI 
weeks claimed varied for different industries and different years, we derived an average annual 
UI weeks claimed that was weighted by the industry composition and the training completion 
date (before 1994, during 1994, after 1994) composition of the trainees. Table 10 shows the 
average annual UI weeks claimed by workers in similar industries to the Retrainees in 1993, 
1994, and 1995. As an aside, the reason that the UI weeks claimed rises slightly between 1994 
and 1995 is simply a product of the SIC weighting of the experience of the Control Group. The 
unweighted UI weeks claimed by newly unemployed in the Control Group actually dropped 
slightly between 1994 and 1995 (unweighted UI weeks were 1993: 21.3 weeks; 1994: 17.1 
weeks; 1995: 17.0 weeks). 






Average Annual UI Weeks Claimed 














1 993 Average 


1994 Average 


1995 Average 


Weighted Average 


Annual UI Weeks Claimed 


21.6 


17.4 


17.5 


18.0 



The final element of the estimation is the average UI payment that would have been received by 
the Retrainees if they had lost their jobs. The calculations are shown in Table 1 1, where the total 
number of Retrainee completers, or 42,036, is multiplied by the 81.0% of Retrainees completers 
whose jobs were in jeopardy which equals 34,049 threatened Retrainees. We used the average 
weekly UI payments for the Retrainees during the year before training on the assumption that it 
reflects the level of UI payments they would have received had they been laid-off. The average 
UI payment was SI 12 per week, which times the number of threatened Retrainees times the 
average number of weeks unemployed (18.0) yields at total of $68,642,784. We estimate that 
the approximately $68,642,784 would have been drawn on UI funds during the subsequent year 
had 81.0% of the Retrainees completers become unemployed because they did not receive ETP 
training. The UI funding actually paid to these Retrainees averaged $174 during the subsequent 
year for a total of $5,924,526. The difference between what these Retrainees were paid and what 
they would have been paid if laid-off is $62,718,258. However, we have already attributed some 
of this difference to what the Retrainees would have been paid had they not been retrained and 
had followed the same trends in unemployment as the Control Group. That amount was 
$1,978,383 in Table 5, and the proportion associated with Retrainees threatened with 
displacement was 81.0% or $1,600,000. Subtracting this from the $62,718,258 above yields a 
savings of $61,1 15,768 in UI fund savings associated with saving the jobs of Retrainees who 
presumably would have lost their jobs without retraining. These calculations are shown in Table 
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Additional Ul Fund Saving Due To Saving Jobs | 


A : : 


Retrainees Placed 


42,036 


Times the Percentage Threatened with Displacement 


81.0% 


Equals the Retrainees Threatened 


34,049 


Times the Average Ul Payment per Week 


$112 


Times the Average Annual Weeks of Unemployment 


18.0 


Equals the Total Ul Payments They Would Have Received 


$68,642,784 


Minus the Ul Payments They Did Receive 


-$5,924,526 


Minus 81% of the Reduction in Ul Payments Due to Increased Job Stability 


-$1,602,490 


Equals the Ul Fund Savings Attributed to ETP Training Saving Jobs 


$61,115,768 



• Outcome 3b: Saving Jobs Prevents California Earnings Losses — Approximately $167 
million 

The second benefit of saving jobs involves preventing Retrainee earnings losses. There are, 
however, two different levels of benefits from avoiding these earnings losses — one is to the 
Retrainees, the other is to the California economy. A total of 34,049 Retrainees completers were 
threatened with job loss during the study period. The estimated value to the Retrainees of 
preventing displacement would be their average weekly wages times the additional weeks that 
they would have been unemployed during the subsequent year. Table 10 shows the expected 
annual weeks of unemployment given displacement was 18.0 weeks. Since the Retrainees 
averaged 1.5 weeks of unemployment during the year after training, the additional weeks of 
unemployment associated with displacement would be 16.5 weeks. Their average weekly wages 
in their year before training was $460. Therefore the expected loss of earnings to the Retrainees 
would be 16.5 weeks times $460 per week times the number of threatened Retrainees (34,049), 
or $258,431,910. However, not all of these Retrainees worked in basic industries so not all of 
their lost earnings would be lost to the California economy. If the Retrainees in nonbasic 
industries had not been trained and had suffered temporary job losses because their companies 
were not able to meet their competition, sales, jobs, and earnings would have decreased in those 
firms but would have expanded sales, jobs, and earnings in their California competitors. While 
there would have been a loss of earnings to the particular workers who did not get training and 
temporarily lost their jobs as a result, the nonbasic jobs and earnings would not have been lost to 
the California economy. 

The lost earnings for the California economy would be the jobs and earnings lost by those 
Retrainees working ir. basic industries because those lost jobs and earnings would have been 
picked up by out-of-state competitors. There were 22,043 Retrainee completers in basic 
industries that were threatened with job loss. The estimated value of this loss of earnings to the 
California economy is the average weekly earnings (pre-training) times the additional annual 
weeks of unemployment associated with displacement times the number of affected workers. 

That is, the expected earnings loss for the California economy during the first year after training 
was 16.5 additional weeks of unemployment times $460 average weekly earnings time 22,043 
Retrainee completers in basic industries, or $167,306,370. 
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Note that estimate of a possible loss in earnings of $167,305,000 presumes that those Retrainees 
in jeopardy of losing their jobs would, in fact, temporarily lose their jobs. Because these jobs are 
in basic industries our analysis also presumes that these jobs would not be lost to other California 
companies, at least not for the typical annual duration of unemployment for these workers given 
a job loss. Our analysis assumes that after the expected 18-week duration, new jobs would 
develop and these workers find them or other suitable jobs. 



• Outcome 3c: Saving Jobs Prevents Multiplier Losses — Approximately $134 million 

The loss of $167,305,000 in earnings in California’s basic sector implies that there would be 
some indirect or multiplier effects of the lost earnings. The logic is that basic sector jobs involve 
supply activities for job materials and services as well as consumer expenditures by the basic 
jobholders. So if the jobs and earnings disappear, even temporarily, the local economy will 
suffer loss of sales by supplying firms and by consumer-oriented producers. The indirect part of 
the multiplier derived above is . 8 , and applying this indirect effect to the estimated earnings loss 
of $167,305,000 yields $133,845,000 in indirect effects on the California economy. The sum of 
the direct and indirect effects of the temporary job losses in basic industries in California 
prevented by ETP retraining of threatened workers is estimated at $301,150,000 for the study 
period. 



• Two Trends in ETP’s Impact on the Economy 

We noted two trends in the magnitude of California job saving effects from ETP training. First, 
beginning January 1, 1994 due to new legislation, roughly 90% or more of Retrainees had to be 
in basic industries. This represents an increase from the percentage of Retrainees in basic 
contracts, which average in the 60-65% range before this legislative change. The increased 
percentage of Retrainees in basic industries should increase the magnitude of job saving benefits 
for the California economy. However the second noted trend moves in the opposite direction. 
Over 95% of the Retrainee completers in the first year of the study period fell under the previous 
legislative requirement that all retraining projects involve workers threatened with displacement 
if they were not retrained. However, just over 55% of Retrainee completers in the second year 
were under contracts that involved a displacement threat. The new legislation effective January 
1, 1994, also allowed other criteria in retraining contracts, including transitioning to a high- 
performance workplace or diversifying the company’s product line. Whether ETP can claim to 
prevent substantial losses in UI funds and in Retrainees’ earnings in the future will depend 
critically on the balance of these trends. During the study period, 52.4% of Retrainee completers 
were threatened workers in basic industries. This percentage is down from previous studies, 
mainly because all the previous retraining contracts had to involve threatened workers. If the 
percentage of retrainees threatened by displacement in ETP contracts stays at the level it was in 
the 1995-96 contracts or continues to fall, the Retrainee earnings losses and associated 
expenditures in the California economy that ETP can claim to have prevented through retraining 
will fall. 
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Impact of Training on Company Growth 



• Overview 

In this section we examine the impact of ETP-funded training on the companies who received 
the training. The fieldwork strongly suggests that ETP-funded training is beneficial to 
companies. That observation derives from the case studies presented in our earlier report, ETP 
at Work. This section examines the impact of ETP-funded training by the statistical analysis of 
the employment and wage growth of companies served by ETP in 1995-1996. This statistical 
analysis confirms what we observed from the fieldwork. 

This analysis compares the growth in total employment and total wages paid of companies with 
ETP training with similar companies who did not have ETP training. We chose these two factors 
because growth in these factors is associated with company success and because uniform data 
were available for all companies served by ETP and for a comparable group of other California 
companies. 

Our analysis leads to the following findings: 

1 Companies with ETP funded training grew faster in terms of total employees and 
total wages paid than similar companies from the same industry. 

2. Companies, which had ETP training through direct contracts (Employer Contracts), 
grew faster than companies that received ETP training through either consortia or 
training agencies. 

3. Employee growth varied with the proportion of employees trained. Companies in 
which 10 to 50% of workers were trained grew faster than companies in which less 
than 10% of workers were trained. Training more than 50% of workers seemed to 
yield about the same growth as training 10% to 50%. 



• Research Approach 

S Growth as a performance measure 

ETP’s mission is to provide training to help California business “remain viable” and “compete in 
the global economy.” To provide a concrete measure of ETP’s ability to achieve these goals we 
attempted to determine whether ETP-funded training promoted growth in the companies it 
served. We assumed that if companies grew more than others in the industry did they were more 
viable and better to able to compete. Specifically we focused on the change (growth) in the 
trainee companies’ number of employees and total wages paid from before-to-after training. 
Growth, or rate of change, measures are more informative than simple numbers. For example, if 
a company had 100 employees before training, an increase of 100 employees would represent a 
100% increase, whereas if the company had 10,000 employees before training, it would represent 
only a 1% increase. 
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S Industry-adjusted growth 

We go beyond looking at simple growth numbers to make comparisons with industry norms. For 
instance, if a firm experienced a 25% increase in employment and its industry experienced an 
average employment growth of 50%, then that company’s 25% growth was poor. However, if 
the industry grew at a 10% rate, then a 25% growth rate is impressive. Thus, we also examine 
industry-adjusted changes. Industry adjustments filter out overall changes in the economy and in 
the trainee firms’ industries and provide a clearer measure of the impact of training. 

To create a comparison group of companies, we selected nine ‘control’ companies for each 
company that received ETP training. These companies were the nine California-based firms 
closest to each trainee company based on size and industry (3-digit SIC code) that did not receive 
ETP training during the same period. The industry average is the average of the nine chosen for 
each trainee company. An industry average provides a benchmark against which to compare the 
trainee company’s characteristics. Since the trainee firms differ from their industry peers in 
terms of ETP training, then differences between trainee firms and their industry peers should 
reflect company-specific ‘abnormal performance’ associated with training. 

Industry-adjusted measures were created by subtracting the industry average’s value from the 
trainee firm’s value. Industry-adjusted performance is informative because it adjusts for the 
many factors that can affect the individual firm values. For instance if we wanted to know 
industry- adjusted wage growth, it would be calculated as trainee firm wage growth minus 
industry average wage growth. The difference would reflect the trainee company’s performance 
after general industry factors are removed. For instance, if a company’s wage growth is 15% and 
the industry s average growth rate was 25%, then the 15% looks poor, resulting in an industry- 
adjusted growth of-10%. On the other hand, if industry wage growth was 5%, then a 15% 
growth rate yields an industry-adjusted growth of +10%. 

S Event time 

We examine company performance in terms of event time. This allows us to view ETP-funded 
training as a treatment and we want to determine whether the treatment affected the trainee 
companies. Since the treatments occurred at different times, we standardize time relative to the 
training period, which is considered to be time zero. All other time is measured by quarter years, 
both before training and after training. The time periods of especial interest are the first year 
before and the first year after training. The period before training tells us what the firms were 
like just before receiving training. We examine impact on the company by comparing the 
characteristics of the company before training to the same characteristics the year after training, 
and comparing their experience to similar companies that did not receive ETP training during the 
period. 
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S Data 



Because most of the firms we studied are private and detailed financials could not be obtained, 
we were limited to two performance measures, number of employees and total wages paid. 

These data were uniformly available from the California Employment Development Department. 
They are collected quarterly as part of the unemployment insurance system. 

The sample of trainee companies is drawn from those that had 10 or more employees receive 
ETP-funded training in 1995-96. We did not believe it was reasonable to expect companies 
which had fewer than 10 people trained to have a significant change in their overall growth as a 
result of training. 

The following definitions show how the variables were calculated: 

Total Earnings - average of quarterly total wages paid to all employees by a given 
company the four quarters before and the four quarters after training, as reported each 
quarter to EDD. 

Total Employees - average of quarterly total number of employees for a given firm the 
four quarters before and the four quarters after training, as reported quarterly to EDD. 

Total Earnings per Employee - total wages divided by total employees for a given firm, 
calculated using the above mentioned variables. 

Total Earnings Growth - change in total wages paid reported for the year (four quarters) 
before training to the year (four quarters) after training. 

Total Employee Growth - growth in number of employees for a company from the year 
(four quarters) proceeding training to the year (four quarters) following training. 

Total Earnings per Employee Growth - growth in earnings per employee for a firm from 
the year proceeding training to the year following training. 

Employer Contract - training for which there was a contract between the individual 
employer and ETP, these are also referred to as ‘stand-alone’ training. 

Consortia Contract - training in which ETP contracts with an industry association or 
major employer to trainee employees from several companies. 

Training Agency Contract - training in which ETP contracts with a training agency to 
provide training to employees from several companies. 
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Growth rate measures were calculated as the change in the company from the year before 
training to the year after training. It is calculated as: 



Year After Year Before 

Growth = (0+1 + 0+2 + 03 + 0+4) - (0-1 + 0-2 + 0-3 + 0-4l 
(Q-l + Q-2 + Q-3 + Q-4) 

Year Before 

Earnings were adjusted for inflation to 1995 dollars. 

We dropped trainee firms that showed 100% decline since this suggests that they no longer 
existed or the data were missing for them. We also dropped 3 firms that showed employment 
growth over 600% since this suggests that they underwent a merger or acquisition, thus 
rendering themselves to be different firms, and therefore not comparable with their past or to 
their industry peers. 

S Company Characteristics 

Table 1 provides details on the characteristics of the average company before it began ETP- 
funded training. The data show that, despite matching ETP companies to the nine companies 
closest in size within their industry, ETP companies are substantially larger on average. That is 
because a few ETP companies are much larger than any other company in their industry. This 
pattern makes the median statistic a better comparison measure than the mean. Examining the 
median figures we see that the typical ETP company-employed 200 people, had a quarterly 
payroll of about $1.8 million or about $9,100 per employee. Control firms were similar if a bit 
smaller overall. 
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Maximum 
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ETP Trained Finn 
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24,527 
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Industry Average 
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7,860 
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ETP Trained Firm 


177 


$8,973,298 


$28,393,400 


$1,849,860 


$322,626,000 


$79,194 


Industry Average 


* 


$4,202,157 


$10,668,100 


$1,489,045 


$104,169,000 


$110,424 


Quarterly Average Total 1 
Earnings per Employee 
the Year before ETP 
Training I 
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ETP Trained Finn 


177 


$9,487 


$3,640 


$9,111 


$25,711 


$1,698 


Industry Average 


# 


$9,251 


$2,696 


$9,110 


$17,084 


$3,311 



* Industry average is calculated as the average of the nine California-based firms closest to trainee firm based on 
size and industry (3 -digit SIC). Hence industry figures represent an average of averages. 
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• Comparison of ETP Companies and Industry Controls 

Companies served by ETP grew substantially faster than similar companies in their industry. 
Table 13 shows that ETP-trained companies grew faster in terms of the number of workers 
employed, and the total earnings they paid out, but were about the same in earnings per 
employee. The average ETP trained firm saw its employment rise by 14.3% from the average 
quarterly employment over the year before training to the year after. This compares highly 
favorably with the .8% decline in the control firms over the same period. Likewise, the trainee 
firms paid out 25.8% more in wages compared to an increase of 10.2% in the control firms. 
Further in ETP trained company payroll per employee rose by 1 1.9% compared to 1 1.7% in the 
control firms. Earnings per employee varied by little between the two groups because the control 
firms, those similar to the trainee firms except for not receiving training, saw their work forces 
decline slightly whereas the trainee firms grew much more rapidly causing their rising payrolls to 
be spread out among more people. Further, firms that grow rapidly tend to add lower paid, 
entry-level workers to the workforce base. 

We found that of the 169 Trainee firms, 57% had employee growth rates that exceeded their 
industry peers by more than 1% and 40% had growth rates 1% or more lower than their industry 
peers, and about 3% of the trainee firms, were within plus or minus 1% of their peers’ growth 
rates. Thus we can say confidently that most ETP firms grew faster than similar companies in 
the same industry. We observed almost the same results with respect to growth in earnings paid. 
The numbers were 56% experienced greater than 1% higher growth, 41% experienced growth at 
least 1% lower, and about 3%, experienced similar growth. With respect to earnings per 
employee, only 41% experienced growth exceeding their industry peers by more than 1%, 53%, 
experienced growth rates at least 1% less than their industry peers, and 6% experienced similar 
growth rates. 

The results on employee growth deserve more comment. Much of our sample is composed of 
manufacturing firms, which have experienced very little employment growth and are expected to 
grow only slightly in the near future according to a recent forecast by UCLA. The small decline 
of the control firms, -.8%, over the period before training to after training comports with this. 

That ETP trained firms employment levels grew at a 14.3% rate over the same period, in spite of 
an overall no growth trend in manufacturing, provides strong evidence that ETP is doing its job. 
Though we cannot ascribe causality, two possible explanations arise that relate to the high 
growth of ETP firms, either of which suggests that ETP is doing its job. Either ETP training is 
causing these companies to grow, or has targeted its money on companies poised for growth. In 
all likelihood both phenomena are occurring. In either case ETP is contributing to overall 
economic growth in the state. 




Training That Makes a Lijference 



60 



53 






v\ ■' 'Table 13;> ; ). 

Descriptive Statistics of Growth for All trainee Firms arid Industry Quarterly Averages: 
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ETP Trained Firm 
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ETP Trained Firm 


169 


25.8% 


50.8% 


15.0% 


241 .9% 


-73.2% 


Industry Average 
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10.2% 


28.2% 


9.0% 


107.9% 


-57.0% 
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ETP Trained Firm 


169 


11.9% 


15.1% 


8.8% 


63.7% 


-35.1% 


Industry Average 


* 


11.7% 


12.9% 


11.9% 


61 .2% 


-35.1% 
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size and industry (4-digit SIC). Hence industry figures represent an average of averages. 
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S Company Impact by Type of Contract 

We also examined the relationship between the type of ETP contract a through which a company 
received ETP training and the impact on the company. As table 3 indicates companies which 
had direct contracts with ETP had the greatest gains, followed by companies served by consortia 
and, finally, those served by training agencies. 

Table 14, shows that there were differences between the types of contracts in the characteristics 
of the companies they served. The average company that used Training Agency training had 
almost $1 1.9 million in payroll the four quarters before training and employed 1,107 workers 
before training, for an average quarterly earnings per employee ratio of $10,067. It is important 
to remember that we dropped from the analysis companies in which fewer than 10 people were 
trained. Training agencies often serve small groups of workers from many companies. Hence 
the data here do not represent all companies served by Training Agency Contracts. The 
companies receiving Consortia-provided training were smaller. Their payroll averaged $3.1 
million per quarter the four quarters before training, and the average number of employees per 
quarter was 354 for average quarterly earnings per employee of $7,668. Companies with direct 
contracts with ETP were in the middle. They had a quarterly average payroll of $8.4 million and 
employed, on average, 794 workers who earned an average of $9,985 per quarter before training. 
Overall, these firms provided good paying jobs with average per worker pay ranging from 
$30,000 to over $40,000. 
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Table 14 



Contract Type 


Average Total 
Quarterly Earnings 


Average Total 
Employees the Year 
before Training 


Average Total 
Earnings/Employee 


^ 

Number 


Employer 


$8,418,795 


794 


$9,985 


68 


Consortia 


$3,138,463 


354 


$7,668 


32 


Training 

Agency 


$11,887,854 


1,107 


*$10,067 


77 



* Significantly different from Consortia at the p < . 01 level. 



Overall, Table 15 shows that Employer Contracts were associated with the greatest industry- 
adjusted growth from the year before training to the year after training in terms of the wages they 
paid out and number of people they employed, 21% and 19. 1%, respectively. This was followed 
by firms using Consortia Contracts, which experienced payroll and employee growths of 16.9% 
and 18%, respectively. Following them, came firms using Training Agency Contracts. These 
firms saw their payrolls grow by 7.6% and employment grow by 8.1%. The results suggest that 
companies that contract directly with ETP receive the most benefits. These results support 
findings from our earlier report ETP At Work. In that report we suggested that employer projects 
had a greater potential for improving productivity and thus growth. We suggested that consortia 
projects had somewhat less potential and training agency projects had the least. We drew these 
conclusions because we observed that in employer contracts there was the potential for greater 
customization of the training, greater organizational change associated with the training if 
training is on site training and all trainees are part of the same company. We also found in 
employer contracts that there was deeper management commitment to the goals of training. 
Conversely training agency projects where a workers from several employers are taught generic 
skills, often on their own time, in a class made up of people from many different companies and 
industries there was much less potential for improving productivity and affecting growth. 
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; Growth in Trainee Firms from 
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! One Year After Training 
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Earnings/ 
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Employer 


30.1% 


17.5% 


12.1% 


66 


Consortia 


31.7% 


16.7% 


14.0% 


31 


Training Agency 


19.4% 


10.3% 


10.7% 


72 


| Industry-Adjusted Growth in 
j Trainee Firms from One Year 
| before Training to One Year 
i After Training 




' 

1 ! 


•' ' , > - - 
'' - -r ' - - ~'~ 

! 

'iA ' '' « « 

''' . ^ 7 




Employer 


21.0% 


19.1% 


0.5% 


66 


Consortia 


16.9% 


18.0% 


-2.1% 


31 


Training Agency 


7.6% 


8.1% 


0.7% 


72 
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• The Impact of the Proportion of Workers Trained: an Examination of Employer 
Contracts 

Only the Employer ontracts allow us to directly observe the relation between the proportion of 
workers trained and growth. This is because the Consortia and Training Agency contracts do not 
tell us how many employees came from the Trainee Firm under study. Thus, the number placed 
for the contract is the sum of a group of workers from many companies. Table 16 shows that we 
divided the contracts into three groups based on the percent of employees trained. The smallest 
training intervention was those companies trained fewer than 10% of the their workers. The 
medium sized interventions were those that trained 10% to 50% of their workers. The largest 
intervention was those companies that trained over 50% of their workers. Not surprisingly it was 
the smaller companies who trained the highest proportion of their workers. 



Table 16 

■ii 111 1 HI SSI 1 Trainee Firms the Vear before Training 


| 1(1 II 


Percent of 
Employees Trained 


Total Earnings 


Total 

Employees 


Total Earnings/ 
Employees 


Number 


< 10% 


$19,380,300 


1686 


$10,731 


19 


10% -50% 


$6,201,000 


666 


$9,196 


29 


> 50% 


$1,222,000 


135 


$9,401 


20 



Table 17 allows us to examine the impact on performance that the proportion of workers trained 
has on company growth. Generally, the greater the proportion of workers trained in a company, 
the higher the growth from the year before training to the year after. This is especially marked 
for companies training more than 10% of their workers. However, the benefits are not 
proportionate to the increase in the workers trained. Training 10% to 50% of the workers does 
not result in two to four times the amount of growth that training less than 10% does. Moreover, 
training above 50% does not result in much more than training from 10%-50% of the workforce 
at a single time. 
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< 10% 


12.5% 


-3.2% 


15.7% 


19 


10% -50% 


38.5% 


25.0% 


12.5% 


27 


> 50% 


35.5% 


27.0% 


8.5% 
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< 10% 


13.1% 


03.7% 


2.1% 


19 


10% - 50% 


24.9% 


4.7% 


2.1% 


27 


> 50% 


23.4% 


6.0% 


-7.2% 
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• Summary and Conclusion 

Companies with ETP training grow faster than similar companies without ETP training. Growth 
varies by the type of contract, and the proportion of workers trained. We observe that employer 
contracts are associated with the greatest growth, perhaps because they are more customized and 
usually train a larger proportion of the workforce. We also observe that training a larger 
proportion of the workers produces a greater, but not proportionally greater, impact on growth. 
We cannot definitely prove causality, but two things may be at work. Either ETP training causes 
greater than expected growth or else companies with the potential to grow faster choose to use 
ETP. Either way, ETP provides what companies need to expand their workforces and payrolls, 
both of which have positive implications for Californians and the California economy. 
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Summary and Recommendations 



In its mission statement ETP pledges to provide funds for training California ’s workforce in 

the skills necessary for businesses to remain viable and compete in the global economy, while 
providing workers with reasonable wages and secure employment. ” In this evaluation study we 
have measured the degree to which ETP has achieved these goals. Based on what we learned in 
the field work component of this project (reported in ETP at Work) and the statistical analysis of 
ETP’s impact reported here we make recommendations to help ETP move closer to achieving its 
mission. We have organized the recommendations around three broad themes: 

• Increasing ETP’s impact on the California economy; 

• Increasing ETP’s impact on companies; 

• Improving the management of the ETP program. 

We summarize our key findings related to these themes and then make specific 
recommendations. 

Increasing ETP’s Impact on the California Economy 

Our analysis revealed three key elements that determine ETP’s impact on the California 
Economy: 

• the effectiveness of its training, 

• the extent to which resources are focused on retraining in basic industries, and 

• the extent to which ETP retraining is focused on threatened workers. 

1. ETP must strive to continually improve the quality of training offered through its 

contracts. 

Effective training is the foundation for ETP’s impact on the California economy. The 
effectiveness of the training determines whether and how much trainee productivity increases It 
is through increased productivity that ETP enhances California companies’ competitiveness, 
saves California jobs, raises workers’ earnings, and increases their employment stability. The 
greater these productivity increases are, the greater the competitive advantage gained, the greater 
the earnings growth, and the greater the increase in suppliers’ orders — all of which determine the 
size of ETP’s impact on the California economy. Our fieldwork revealed that the elements of 
effective training go beyond the curriculum and the instructors. Effective training must be 
customized to the company and its workers; training must be well planned, skillfully delivered, 
and reinforced by management. Efforts to improve training have big payoffs. More effective 
training means higher earnings, more California job opportunities, and lower unemployment. 
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2. ETP must continue to target its resources on basic industries. 

Basic industries leverage ETP’s impact on the California economy. California’s basic industries 
are those that face out-of-state competition. Thus, ETP retraining in basic industries generates a 
competitive advantage for California companies relative to their out-of-state competitors. This 
leads to more business for both California companies that retrain their workers and the California 
companies that supply these companies and their workers. In this study period, a one hundred- 
dollar increase in basic California business activity from effective retraining generated another 
eighty dollars in business for their suppliers. ETP has recognized this and is required to focus its 
standard retraining projects on companies in basic industries. If ETP were to fund retraining 
projects for nonbasic businesses— ones that just serve local trade — the training-produced gains 
for that California business would come at the expense of one of its competitors down the street. 
There would be no net gain for the California economy in this case. 

We recognize that the Panel is under constant pressure to extend eligibility for ETP training to 
industries that are not basic or may face only limited out-of-state competition. When confronted 
with these pressures the Panel must recognize that expending resources on these industries 
significantly diminishes the economic impact of ETP. 

3. ETP should increase the proportion of its funds targeted on potentially displaced workers. 

Saving California jobs produced ETP’s largest monetary impact on the California economy 
during the study period. Had the jobs that were threatened by out-of-state competition been 
temporarily lost for just the length of the average unemployment episode, California would have 
lost over $362 million in additional unemployment claims, decreased business and earnings, and 
generated negative indirect effects on suppliers. 

Curiously, ETP seemed to de-emphasize retraining programs aimed at saving threatened jobs in 
the second year of the study period. New regulations taking effect at that time allowed for other 
training project objectives (such as moving to a high performance workplace) Perhaps this 
trend” just reflects a more accurate classification of the projects that were never actually aimed 
at saving threatened jobs. Whether it’s a “trend” or not, the point is clear — the fewer California 
jobs that ETP retraining saves from out-of-state competition the less will be ETP’s direct impact 
on the economy. 

After two years, ETP should review trends in the proportion of funds going to potentially 
displaced workers. ETP should also evaluate its policies and practices to see if the Panel has 
been able to reliably identify potentially displaced workers, and determine if the benefits 
projected for this policy have been realized. 

Later we recommend that ETP target companies whose operations are below the industries best 
practices. This recommendation dovetails with that strategy because it is the workers in these 
low performing companies that are most at risk for losing their jobs. 
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Increasing ETP’s Impact on Companies 

This project broke new ground in evaluating ETP’s impact. First, we used case studies to 
examine ETP’s impact on 23 companies. Next, we used data from the UI system to measure the 
impact of ETP training on the growth of 177 companies. Both analyses showed that ETP 
training does have a positive impact on participating companies. The 23 case studies also 
enabled us to explore the dynamics of ETP training to find factors that affected the impact of 
ETP training on companies. Based on that analysis we offer the following recommendations. 

4. Target ETP training on companies that are below industry best practices. 

This may seem a counter intuitive recommendation to many. Panel members and staff may wish 
to work with companies that are well-managed industry leaders, rather than less well-managed 
struggling companies. However, our fieldwork found that companies whose operations were far 
below industry standards had the greatest potential gains from ETP training. Similarly, workers 
in these low performing companies are most at risk for losing their jobs, and saving those jobs is 
the most efficient way for ETP to have a positive economic impact on the state. 

To implement this recommendation, ETP will need to train its staff to identify companies 
operating below industry standards. This can be done two ways. First, the companies could 
provide data benchmarking their performance in areas such as scrap rates, labor productivity, 
warrantee returns or customer satisfaction against industry leaders to show how far away they are 
from the best practices. Second, companies could document a lack of organizational 
characteristics associated with high performance workplaces. A recent review of the research on 
high performance workplaces found five separate innovations associated with high performance 
work places for which ETP staff could check (Kirkman and Lowe, 1998). These innovations 
are: 

• Self-managing work teams, 

• Employee involvement, participation, empowerment, 

• Being a learning organization, 

• Integrated production technologies, 

• Total quality management. 

The more of these innovations that are absent the greater the potential gains of implementing one 
or more of the innovations. 

One paradox in pursuing this strategy is that it means ETP must deal with the companies and 
managers who are least well prepared to manage the ETP process. This may mean investing 
additional staff time in developing proposals and managing projects, but it should lead to a 
greater impact on the companies served. 

5. ETP should encourage management reinforcement of training. 

Our fieldwork showed that even poorly-planned, low-quality training could have a positive 
impact on companies if management provided strong reinforcement of the skills and techniques 
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learned. For example, even low quality generic TQM training, could lead to large productivity 
improvements if management reinforced training by allocating time for quality teams to work 
and empowering them to make changes. Conversely, well-planned, high-quality training often 
failed to yield positive results if there was not methodical management reinforcement. 

ETP now attempts to promote management commitment to training by requiring high level 
managers to sign-off on contracts. ETP needs to go further and require applicants to document 
their plans for reinforcing training and implementing related changes in production systems and 
technology. This may require providing ETP staff, training agency staff, consultants and 
employers with training on how to reinforce training. We found three factors that characterized 
effective reinforcement that could serve as a basis for such training: 

• Positive messages from management about the value of training, before, during, and 
after training, 

• Reinforcement of newly acquired skills and techniques, during and after training, 

• Prompt reinforcement by the implementation of new production techniques and 
technology immediately after training. 

ETP could promote better reinforcement of training by providing contractors and consultants 
with case examples of model reinforcement efforts in successful ETP projects. These cases 
could include example of how managers have tied compensation to implementation of process 
reforms, provided career growth after training, and gave workers larger input into decision 
making, or infused new technology in the production process. 

6. Increase the amount of customized employer-provided training and reduce the amount of 
generic, training-agency training. 

In this project, we examined the impact of various types of ETP contracts, employer contracts, 
consortia contracts and training agency contracts. We also examined contracts in which training 
was company specific, industry specific or generic. Both the field work and the analysis of the 
impact of ETP training on company growth reveal that the most effective training model was the 
employer contract where a company provides customized training to its own workers. The least 
effective training projects were training agency contracts where workers were trained often off- 
the-clock, in generic-skills such as training in Microsoft Office. Our analysis shows that in 
recent years, ETP funding has shifted away from the customized, employer-provided training to 
the generic skills, training agency contracts. For example, training agency contracts served 20% 
of all trainees in 1994-95. In 1995-96 they served 30% of all trainees, a growth rate of 50%. If 
ETP wishes to increase the impact of ETP training on companies we recommend this trend be 
reversed. Thus we make the following specific recommendations: 

6a. Do not fund training where employers are not paying a significant share of the 

costs. 

One reason ETP has showed a greater impact than other public training programs, is that it 
closely resembles private employer training programs. This is because the decision to enter an 
ETP program has been very similar to a decision to provide private training. The decision was 
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similar because employers had to pay the employees for the time they were in training and bear 
the costs of reduced production while they completed SOST assignments. Now ETP is funding a 
significant number of projects where trainees are “off the clock,” attending training after hours 
and without compensation at a location provided by a training agency. Since this training is truly 
free to employers, unlike traditional ETP projects, we found that employers were much less 
involved in the training. The training was less customized to the needs of employers, and 
employers were much less selective in who they chose to send to training. Employers were 
much less aware of what the trainee was learning in training and less likely to reinforce the use 
of new skills when the trainee returned from training. All these factors combine to reduce the 
impact of the training. In fact, in many cases, the programs were no different from standard 
evening courses offered by adult schools, community colleges or university extensions. If 
employers must pay employees for training time, or bear other costs such as supplies, equipment, 
or instructors of training facilities, they shop much more carefully and will be more likely to 
only commit to training that has value for the company. 

An argument made in favor of this arrangement is that small employers cannot afford to bear the 
costs of training. Our analysis found that many participating employers in these programs were 
not small. We also noted that labor law requires that employees be paid for mandatory training. 

In our view, if small employers want employees to voluntarily attend generic training without 
compensation, they can make use of existing low cost public evening programs, which are 
widely available. It does not appear to us that it is ETP’s mission to compete with this vast 
sector of public programs. 

In employer contracts ETP has begun to collect data on employer’s in-kind contribution. We 
recommend that in training agency contracts, if employees are “off-the-clock” for training, 
employers be required to make a substantial contribution toward the cost of training. These 
contributions could be in the form of equipment, training, facilities, staff serving as instructors, 
or providing training materials. As an incentive to seek effective employee training, it seems 
reasonable to require employers to bear 20 - 40 percent of the cost of training. 

6b. Over two years phase out generic skills training and focus on industry specific 

skills training. 

The fieldwork turned up two types of training agency projects that the Panel needs to distinguish 
between in making policy. The first are projects that taught “generic skills.” The best examples 
of these generic skill projects are programs that taught Microsoft Office. Trainees are taught in 
classes made up employees from several different employers, and several industries. No specific 
applications of the packages are taught. All trainees are taught all the components of the 
package regardless of what is required on the job. The second type of program we labeled 
“industry-specific skills training.” These projects provide skills that are applied to specific jobs 
in a given industry. The best example of this type of skill was a program that taught printed 
circuit board design. This training is very specific to the electronics industry and is not readily 
available in the public sector. In evaluating programs we particularly recommend that the Panel 
focus on projects that train in bottleneck occupations. For example, California is currently facing 
a shortage of qualified machinist and toolmakers. Training that trains employees in these skills, 
while not company specific, is still of much more value than generic skills training. In many 
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cases, the training takes trainees out of a low paying career track to one that puts them solidly 
above average. 

Recognizing that a number of agencies have built up large programs based on ETP funding of a 
generic skills training, we recommend that the training be phased out rather than stopped 
immediately. This will allow time for the agencies and the employers they serve to make the 
adjustment needed to best serve the workers and taxpayers of California. 

7. ETP should not require that a minimum or maximum proportion of a company’s 
workers be trained. 

We explored whether increasing the proportion the workforce trained was associated with an 
increased impact of training on the company. While we found a limited positive association 
between the percent of workers trained and the impact on companies, the increase in impact was 
not directly proportional to the increase in employees trained. It does not appear that there is a 
minimum proportion of workers that must be trained for ETP to have a positive impact, nor does 
there appear to be a point at which the impact of ETP training dramatically diminishes as the 
proportion trained rises. Hence we do not recommend setting a minimum or maximum 
proportions of workers to be trained. This is probably a decision best left to employers. 

This is an area where further research would be productive. It may be with certain types of 
innovations, such as TQM, that there is a minimum proportion of employees that should be 
trained. It seems to us that a research approach that combined careful case studies with 
quantitative measures of the impact of training on the company could yield a more detailed 
understanding of this issue and enable ETP to better invest its funds. 

8. ETP should continue to attempt to measure the impact of ETP training on individual 

companies. 

Our attempt to measure the impact of ETP training on companies reveals both the problems and 
promise of this approach. Despite a wide spread interest in the impact of training on company 
performance there are few published studies due to the many difficulties in getting performance 
data on companies and finding training interventions of sufficient size that they could reasonably 
be expected to affect company performance. 

From our effort, we learned that it is possible to obtain valid and reliable company-level 
employment growth data, that can be accurately benchmarked against similar companies. 
Unfortunately, detailed company-level data was not uniformly available for all ETP and 
comparison companies. Nevertheless, our analysis generated valuable insights into ETP’s 
performance and the nature of various types of ETP contracts. 

We recommend that ETP explore ways to collect the company growth data on a regular basis for 
ETP companies and a comparison group to provide a regular method monitoring ETP’s impact 
on the companies served. 
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Improving the Management of the ETP Program 

9. Continue to create incentives and sanctions to ensure complete and correct data 

reporting. 

Our ability to measure the impact of ETP training was significantly hampered by missing data 
which should have been collected on program participants. The data files for the New Hire 
trainees had so much missing data that we could not use 80% of the New Hire records in the 
regression analysis. That is, out of 4,336 trainee records, we had to delete 3,462 from this 
analysis due to missing data, using only 874 records in our search for statistically significant 
relationships. The Retrainee records fared only somewhat better in the final analysis, with only 
44% of the records discarded because of missing data (23,837 out of 53,121). This is a new 
problem. In the past, complete data were available on about 90% of participants. ETP needs 
these data to monitor its programs and measure its performance. It appears that during the period 
studied, providing demographic information was voluntary, causing the low response rate we 
found. Currently, ETP is again requiring complete demographic data for participants. 
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Comparison of Trainees 
Before 1994, 1994, and After 1994 
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TABLE B— 1 

REGRESSION RESULTS FOR RETRAINEES USING ALL INDEPENDENT 
DEMOGRAPHIC AND PROGRAMMATIC VARIABLES 



Dependent Variables: YADJ1YR4 N: 29284 Multiple R: 0.03 Squared multiple R: 0.00 
Adjusted squared multiple R; 0.00 Standard error of estimate: 4485290.69 



Effect 


Coefficient 


Std Error 


Std Coef 


Tolerance 


t 


P (2 Tail) 


CONSTANT 


-1 . 84645E+08 


1.81259E+08 


0.00 




-1.02 


0.31 


TRAINING COMPLETION 


16856.72 


95807.85 


0.00 


0 . 47 


0.18 


0 . 86 


ETHNICITY 1 


-139079.39 


311577.00 


-0.02 


0.03 


-0.45 


0.66 


ETHNICITY 2 


-130497.41 


330613.79 


-0.01 


0.13 


-0.39 


0.69 


ETHNICITY 3 


-226077.69 


315421 . 89 


-0.02 


0 . 04 


-0.72 


0 .47 


ETHNICITY 4 


-234781.62 


421536.03 


-0.00 


0.46 


-0 . 56 


0.58 


ETHNICITY 5 


-124149 . 00 


322047.26 


-0.01 


0.08 


-0 . 39 


0 . 70 


ETHNICITY 6 


-232136.73 


529607.47 


-0.00 


0.66 


-0.44 


0.66 


ETHNICITY 7 


-146087.20 


339816.29 


-0.01 


0.16 


-0.43 


0 .67 


ETHNICITY 8 


-68287.12 


359988.23 


-0.00 


0.27 


-0.19 


0 . 85 


ETHNICITY 9 


-236629.99 


416681.77 


-0.00 


0.44 


-0.57 


0 . 57 


CONTRACT TYPE 1 


-36768.29 


176621.41 


-0.00 


0.18 


-0.21 


0 . 84 


CONTRACT TYPE 2 


-45497 . 10 


207998 . 89 


-0.00 


0.10 


-0.22 


0 . 83 


SOFT SKILLS 


87292 . 36 


103517.96 


0.01 


0.36 


0.84 


0.40 


BASIC SKILLS 


-103282.02 


77101 .78 


-0.01 


0.62 


-1.34 


0.18 


HARD SKILLS 


31612 . 21 


64890.91 


0.00 


0.71 


0.49 


0 . 63 


AGE 


9238133.94 


9063437.36 


22.07 


0.00 


1.02 


0 . 31 


EXPERIENCE 


-9235632.44 


9063451.85 


-22.06 


0.00 


-1 . 02 


0 . 31 


NUMINBU2 


-12237.96 


99797 . 11 


-0.00 


0.58 


-0.12 


0 . 90 


NUMINBU3 


-71497 .20 


78354.60 


-0.01 


0.46 


-0.91 


0.36 


NUMINBU4 


-70172 .29 


400024.08 


-0.00 


0.97 


-0.18 


0 . 86 


NUMINBU5 


-2383.74 


1299801.20 


-0.00 


0 . 99 


-0 . 00 


1 . 00 


NUMINBU6 


-365553.76 


1590157.65 


-0.00 


0 . 99 


-0.23 


0 . 82 


NUMINBU7 


-80329.06 


4487242 . 93 


-0.00 


1.00 


-0.02 


0.99 


LITERACY HOURS 


824.21 


2486.02 


0.00 


0.86 


0 . 33 


0 . 74 


LABORATORY HOURS 


-413 .40 


1201 . 71 


-0.00 


0 . 77 


-0 . 34 


0 . 73 


SOST HOURS 


-256 . 91 


658.79 


-0.00 


0.62 


-0 . 39 


0 . 70 


CLASS HOURS 


-137.90 


599.89 


-0.00 


0.72 


-0.23 


0 . 82 


GENDER 


3112 . 12 


59373.95 


0.00 


0 . 80 


0.05 


0 . 96 


EDUCATIONAL LEVEL 2 


58572.84 


172746 . 94 


0.00 


0.41 


0 . 34 


0.73 


EDUCATIONAL LEVEL 3 


-53722.15 


142036.04 


-0.01 


0 .17 


-0.38 


0 . 71 


EDUCATIONAL LEVEL 4 


-1914 . 74 


248012.01 


-0.00 


0 . 71 


-0 . 01 


0 . 99 


EDUCATIONAL LEVEL 5 


7798 . 12 


141925.46 


0.00 


0.15 


0.05 


0 . 96 


EDUCATIONAL LEVEL 6 


-56349 . 32 


147731 . 09 


-0.01 


0.20 


-0 . 38 


0 . 70 


EDUCATIONAL LEVEL 7 


-234321.54 


172531.39 


-0.01 


0 .41 


-1 . 36 


0 . 17 


MARITAL STATUS 


47079 . 24 


56000.87 


0.01 


0.94 


0 . 84 


0.40 


VETERAN STATUS 


-27556.66 


90794.03 


-0.00 


0.82 


-0 . 30 


0 . 76 


DISABLED STATUS 


-44129.78 


284140.55 


-0.00 


0.99 


-0 . 16 


0 . 88 


GAIN PARTICIPANT 


318323.51 


324243.68 


0.01 


0.99 


0 . 98 


0.33 


APPRENTICESHIP 


88235.39 


249323.54 


0.00 


0.86 


0.35 


0.72 


BUSINESS TYPE B 


-45993 . 27 


508124.74 


-0.00 


0 .17 


-0 . 09 


0 . 93 


BUSINESS TYPE C 


65754 . 97 


461023.70 


0.01 


0.01 


0.14 


0 . 89 


BUSINESS TYPE D 


69139 . 85 


481781 . 93 


0.00 


0.08 


0 . 14 


0.89 


BUSINESS TYPE E 


-95849.58 


467295 . 98 


-0.01 


0.03 


-0.21 


0 . 84 


BUSINESS TYPE F 


35394.62 


464086.18 


0.00 


0.03 


0 . 08 


0 . 94 


BUSINESS TYPE G 


53685.51 


458062.72 


0.00 


0.02 


0.12 


0 . 91 


BUSINESS TYPE H 


29289.53 


497555 . 94 


0.00 


0.15 


0 . 06 


0 . 95 


BUSINESS SIZE 2 


-67226.76 


331908.28 


-0.00 


0.21 


-0.20 


0.84 


BUSINESS SIZE 3 


-60485.04 


312912.99 


-0.00 


0 . 08 


-0.19 


0 . 85 


BUSINESS SIZE 4 


-34680.21 


301196.90 


-0.00 


0.03 


-0.12 


0 . 91 


BUSINESS SIZE 6 


63046.17 


356624.76 


0.01 


0.02 


0.18 


0.86 
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TABLE B— 2 

REGRESSION RESULTS POR NEW HIRES USING ALL INDEPENDENT 
DEMOGRAPHIC AND PROGRAMMATIC VARIABLES 



Dep Var: YADJ1YR4 N: 874 Multiple R: 0.20 Squared multiple R: 0.04 
Adjusted squared multiple R: 0.00 Standard error of estimate: 1252817.77 



Effect 




Coefficient 


Std Error 


Std Coef 


Tolerance 


t 


P (2 Tail) 


CONSTANT 




-8973874 . 68 


2 . 97253E+08 


0.00 




-0.03 


0 . 98 


TRAINING COMPLETION 


241553.83 


146224 . 96 


0.10 


0.34 


1.65 


0.10 


ETHNICITY 1 




-420246.06 


1285942.85 


-0.16 


0.00 


-0.33 


0 . 74 


ETHNICITY 2 




-389449 . 00 


1285424 . 10 


-0.12 


0.01 


-0.30 


0.76 


ETHNICITY 3 




-574153.77 


1282442 . 58 


-0.21 


0.01 


-0.45 


0.65 


ETHNICITY 4 




-666034.17 


1359613.84 


-0.05 


0.11 


-0.49 


0.62 


ETHNICITY 5 




-566666.89 


1297301.22 


-0.12 


0.02 


-0.44 


0.66 


ETHNICITY 6 




-910161.42 


1431962.90 


-0.05 


0.19 


-0.64 


0.53 


ETHNICITY 7 




-601563.58 


1303819.19 


-0.10 


0.03 


-0.46 


0.64 


ETHNICITY 8 
ETHNICITY 9 




-255779.71 


1322243.67 


-0.03 


0.06 


-0.19 


0.85 


SOFT SKILLS 




310004 . 84 


213196.26 


0.10 


0.22 


1.45 


0.15 


BASIC SKILLS 




-36188.00 


150766.41 


-0.01 


0 .44 


-0.24 


0.81 


HARD SKILLS 




-14888 . 08 


164434.39 


-0.01 


0.36 


-0.09 


0.93 


AGE 




281613.77 


14833396.56 


2.37 


0.00 


0 . 02 


0.98 


EXPERIENCE 




-282895.15 


14833187.42 


-2.38 


0.00 


o 

o 

to 


0.98 


NUMINBU2 




4364 . 69 


211535.90 


0.00 


0.84 


0.02 


0.98 


NUMINBU3 




-222704 .06 


163794.79 


-0.07 


0.39 


-1 . 36 


0 . 17 


NUMINBU4 

NUMINBU5 

NUMINBU6 

NUMINBU7 




-1503481.01 


662250.63 


-0.08 


0.90 


-2.27 


0.02 


LITERACY HOURS 




192 . 83 


8027.12 


0.00 


0.91 


0.02 


0.98 


LABORATORY HOURS 
SOST HOURS - 




-402 . 92 


784 . 53 


-0.03 


0.42 


-0.51 


0.61 


CLASS HOURS. 




-73 .39 


301 . 08 


-0.01 


0.42 


-0.24 


0 . 81 


GENDER 




22865.57 


115837 . 01 


0.01 


0.56 


0.20 


0.84 


EDUCATIONAL LEVEL 


2 


570662.80 


303528.18 


0.14 


0.20 


1.88 


0.06 


EDUCATIONAL LEVEL 


3 


115711.47 


271098.72 


0.04 


0.11 


0.43 


0.67 


EDUCATIONAL LEVEL 


4 


376858.56 


316289.45 


0.07 


0.31 


1.19 


0.23 


EDUCATIONAL LEVEL 


5 


-40039.25 


275186.81 


-0.01 


0.11 


-0.15 


0.88 


EDUCATIONAL LEVEL 


6 


8162 .41 


293504.21 


0.00 


0.18 


0.03 


0.98 


EDUCATIONAL LEVEL 


7 


-3768 . 86 


407977.02 


-0.00 


0.57 


-0.01 


0.99 


MARITAL STATUS 




32649.75 


91444 . 91 


0.01 


0 . 86 


0 . 36 


0.72 


VETERAN STATUS 




-28010.17 


138515.70 


-0.01 


0 . 84 


-0.20 


0.84 


DISABLED STATUS 




598104.56 


482594 .43 


0.04 


0 . 97 


1.24 


0.22 


GAIN PARTICIPANT 




269669.55 


208804.95 


0.05 


0.72 


1.29 


0.20 


APPRENTICESHIP 
BUSINESS TYPE B 




-224190.44 


341879.39 


-0.02 


0.85 


-0 . 66 


0.51 


BUSINESS TYPE C 
BUSINESS TYPE D 




49546.72 


226971.90 


0.02 


0 . 14 


0.22 


0.83 


BUSINESS TYPE E 




485037.07 


868698.86 


0.03 


0.52 


0.56 


0.58 


BUSINESS TYPE F 




-65079.47 


306672.93 


-0.01 


0.54 


-0.21 


0.83 


BUSINESS TYPE G 
BUSINESS TYPE H 
BUSINESS SIZE 2 




74368 . 99 


208391.42 


0.03 


0 . 17 


0.36 


0.72 


BUSINESS SIZE 3 
















BUSINESS SIZE 4 




-194927.55 


579738.36 


-0.06 


0 . 04 


-0.34 


0 . 74 


BUSINESS SIZE 6 




131408.54 


467340.15 


0.04 


0.06 


0.28 


0 .78 
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TABLE B— 3 

REGRESSION RESULTS FOR RETRAINEES USING CONTRACT TYPE AND 
SELECTED EDUCATION LEVEL AND BUSINESS TYPE INDEPENDENT VARIABLES 



Dependent Variables: YADJ1YR4 N: 51514 Multiple R: 0.01 Squared multiple R: 0.00 
Adjusted squared multiple R: 0.00 Standard error of estimate: 4751093.12 



Effect 


Coefficient 


Std Error 


Std Coef 


Tolerance 


t 


P ( 2 Tail) 


CONSTANT 


95709.75 


28476.20 


0.00 




3.36 


0.00 


CONTRACT TYPE 1 


-7411 . 11 


67528.94 


-0.00 


0.94 


-0.11 


0.91 


CONTRACT TYPE 2 


-29943.17 


54006.38 


-0.00 


0.92 


-0.55 


0.58 


EDUCATIONAL LEVEL 7 


-165391.34 


83110.21 


-0.01 


1.00 


-1.99 


0.05 


BUSINESS TYPE E 


-120059.13 


67886.26 


-0.01 


0.94 


-1 .77 


0.08 



o 

ERIC 
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TABLE B— 4 

REGRESSION RESULTS FOR NEW HIRES USING CONTRACT TYPE AND SELECTED (NUMINBUS) , 
EDUCATION LEVEL, AND GAIN PARTICIPATION INDEPENDENT VARIABLES 



Dependent Variables: YADJ1YR4 N: 986 Multiple R: 0.14 Squared multiple R: 0.02 
Adjusted squared multiple R: 0.01 Standard error of estimate: 1222048.31 



Effect 


Coefficient 


Std Error 


Std Coef 


Tolerance 


t 


P (2 Tail) 


CONSTANT 


-78994.59 


92903.97 


0.00 




-0.85 


0.40 


CONTRACT TYPE 1 


177092.09 


232133 . 06 


0.03 


0 .87 


0.76 


0.45 


CONTRACT TYPE 2 


69186.90 


103746.40 


0.02 


0.85 


0.67 


0.51 


NUMINBU4 


-1434888 . 26 


617575.09 


-0.07 


0.98 


-2.32 


0.02 


EDUCATIONAL LEVEL 2 


316118.87 


125658.35 


0.08 


0.95 


2 . 52 


0.01 


EDUCATIONAL LEVEL 4 


390921.23 


170522.26 


0 . 07 


0 . 99 


2.29 


0.02 


GAIN PARTICIPANT 


260111 . 86 


157834.99 


0.05 


0 . 99 


1.65 


0.10 
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